{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 机器人自动走迷宫\n",
    "\n",
    "<br>\n",
    "<hr>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "toc-hr-collapsed": false
   },
   "source": [
    "# 1. 实验介绍  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.1 实验内容  \n",
    "在本实验中，要求分别使用基础搜索算法和 Deep QLearning 算法，完成机器人自动走迷宫。\n",
    " \n",
    "<img src=\"https://imgbed.momodel.cn/20200914145238.png\" width=\"40%\"/>\n",
    "\n",
    "如上图所示，左上角的红色椭圆既是起点也是机器人的初始位置，右下角的绿色方块是出口。          \n",
    "游戏规则为：从起点开始，通过错综复杂的迷宫，到达目标点(出口)。\n",
    "        \n",
    "+ 在任一位置可执行动作包括：向上走 `'u'`、向右走 `'r'`、向下走 `'d'`、向左走 `'l'`。\n",
    "\n",
    "+ 执行不同的动作后，根据不同的情况会获得不同的奖励，具体而言，有以下几种情况。\n",
    "    - 撞墙\n",
    "    - 走到出口\n",
    "    - 其余情况\n",
    "    \n",
    "    \n",
    "+ 需要您分别实现**基于基础搜索算法**和 **Deep QLearning 算法**的机器人，使机器人自动走到迷宫的出口。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.2 实验要求 \n",
    "+ 使用 Python 语言。\n",
    "+ 使用基础搜索算法完成机器人走迷宫。\n",
    "+ 使用 Deep QLearning 算法完成机器人走迷宫。\n",
    "+ 算法部分需要自己实现，不能使用现成的包、工具或者接口。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.3 实验环境\n",
    "可以使用 Python 实现基础算法的实现， 使用 Keras、PyTorch等框架实现 Deep QLearning 算法。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.4 注意事项\n",
    "+ Python 与 Python Package 的使用方式，可在右侧 `API文档` 中查阅。\n",
    "+ 当右上角的『Python 3』长时间指示为运行中的时候，造成代码无法执行时，可以重新启动 Kernel 解决（左上角『Kernel』-『Restart Kernel』）。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.5 参考资料\n",
    "+  强化学习入门MDP：https://zhuanlan.zhihu.com/p/25498081\n",
    "+ QLearning 示例：http://mnemstudio.org/path-finding-q-learning-tutorial.htm\n",
    "+ QLearning 知乎解释：https://www.zhihu.com/question/26408259\n",
    "+ DeepQLearning 论文：https://files.momodel.cn/Playing%20Atari%20with%20Deep%20Reinforcement%20Learning.pdf\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "toc-hr-collapsed": false
   },
   "source": [
    "# 2. 实验内容\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "toc-hr-collapsed": false
   },
   "source": [
    "\n",
    "## 2.1 Maze 类介绍"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1.1 创建迷宫\n",
    "通过迷宫类 Maze 可以随机创建一个迷宫。\n",
    "\n",
    "1. 使用  Maze(maze_size=size)  来随机生成一个 size * size 大小的迷宫。\n",
    "2. 使用 print() 函数可以输出迷宫的 size 以及画出迷宫图\n",
    "3. 红色的圆是机器人初始位置\n",
    "4. 绿色的方块是迷宫的出口位置"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "deletable": false,
    "select": true
   },
   "outputs": [],
   "source": [
    "# 导入相关包 \n",
    "import os\n",
    "import random\n",
    "import numpy as np\n",
    "from Maze import Maze\n",
    "from Runner import Runner\n",
    "from QRobot import QRobot\n",
    "from ReplayDataSet import ReplayDataSet\n",
    "from torch_py.MinDQNRobot import MinDQNRobot as TorchRobot # PyTorch版本\n",
    "# from keras_py.MinDQNRobot import MinDQNRobot as KerasRobot # Keras版本\n",
    "import matplotlib.pyplot as plt\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAroAAAHPCAYAAAC8+nn2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAABYlAAAWJQFJUiTwAAASoElEQVR4nO3dT4il2V3G8edkOiGa0RaJ7iQquDNaREFFtBv8E0Eh7gRBISBBhEQQcaWYbFwILsRs/IOYjYLgIupGE6RqoZCF2ErEhSFxFBVU0DJBNCE5Lqpm6EBGu6ar7qn3OZ8PFNObqfmdOee+93vfurdrzDkDAABtXrd6AAAAuAtCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEoPVg/wWowx3pvk51fPcWIXSR6tHuLEdlvzbutNrHkX1rwHa97D++ac7109xLM63B3d68h9vHiMFc5WD8CdO1s9ACdxtnqABc5WD8BJnK0egJN4fN1ih3C40M3VndzdXj0lycPVA3Dn7PEedtznHde8I/u8h0c50E/VD/nWhZfNOcfqGU5hjDFf/vMua96NPd7Djvu845p3ZJ/38PQ+H8WhQ/dOjDGSvDnJ1zz19aVJXp/khSSfSfLpJP+U5BPXXy9lzv9eMi8AAF+Q0B3jQZJvTvI9119vS/Km1/B9/j5Xb0r/UJIPZ85/vsUpAQC4oTHnse5C38qPR67u2j5O8uNJ3p67eV/RR5P8bpJfzZz/8jzfyI+E+tnjPey4zzuueUf2eQ9H3Oe9QneML0ryw0nek+QbbneyV/XpJL+T5Jcz51+8lm9wxIPFzdjjPey4zzuueUf2eQ9H3Oc9QvfqDu6PJvnFJF95N5M9kz9M8pOZ8+M3+ZeOeLC4GXu8hx33ecc178g+7+GI+3zEv17sZsb46iR/kuS3sjZyk+QHkvx1xvipjNH//x4AYKHu2BrjHUn+KvfrF0y8MckvJfnjjPFli2cBAKjVG7pj/FCS30vyJatHeRXfleTDGePLVw8CANCoM3TH+JEkv52rv/f2PvumJH+SMb5i9SAAAG36QneMx0k+kOOs7RuTfDBj3PcoBwA4lKPE4LO5+uvDfi3JIT4J+JRvS/ITq4cAAGjSFbrJzyb5utVDvEa/kDG+avUQAAAtekJ3jDcn+ZnVYzyHF5P83OohAABa9IRu8v1JHqwe4jm9w9+vCwBwO5qi6h2rB7gFX5nkW1YPAQDQoCN0r37F7/euHuOWvH31AAAADTpCN/niJG9aPcQtWf1rigEAKrSE7htWD3CLmtYCALBMS+h+avUAt6hpLQAAy3SE7pyfSfI3q8e4JX+5egAAgAYdoXvlg6sHuAUzyR+uHgIAoEFT6P7+6gFuwZ9lzn9dPQQAQIOm0P1Iko+uHuI5/cbqAQAAWvSE7pyfS/Jjufrx/xGdJ/nA6iEAAFr0hG6SzPmRJL+yeozX4H+SvCtzHjXSAQDuna7QvfKzSf589RA39O7M+berhwAAaNIXunN+Msl35+o9u/fdzNWd3F9fPQgAQJu+0E2SOf8jyfcm+dPFk/xfPpfknSIXAOBudIZuksz5n7mK3fevHuUL+Mckb8+cPnwGAHBHekM3Seb8r8z57iTfk+S+vAf215O8NXN+ePUgAADNukP3ZVdR+dYkP53kH1ZMkOQPknxr5nxX5vz3BTMAAGxlHO1vtBpjvDLwnHO8hm/wIMkPJnlPku+4tcG+sE8m+c0k78+cH3ut3+S518y9Z4/3sOM+77jmHdnnPRxxn/cL3c//Zl+f5Pty9daG70zyxuf6flf+LsmHrr/+6Pq9ws/liAeLm7HHe9hxn3dc847s8x6OuM97h+7nf+M3Jvn2JG9L8jVJvvb6n1+d5A1f4N/4tySfuP76eJKP5eq3m338tn/xwxEPFjdjj/ew4z7vuOYd2ec9HHGfhe7//x98Xa5C9/VJXkjymSSfzpyfufP/9isjHO9gcTP2eA877vOOa96Rfd7DEff5weoB7r05P5fkv6+/AAA4CKEL98zTr5g3cDHnfLx6CE7D2abNGOM8yaPVc/Dqjhi6F6sHWGC7NV9fPJIkmzxZXCQ5S/Jw8RzcrcvVAyxwmeTFXL31aydnY4zzTa5fyb7PU2eLx1jhMsmT1UM8K+/R5V7acZ83vTOw1V2vTc/1sZ5kbtku+7wjZ/sYZ1voci/ZZxrteK6teY8172jHfT7imvf4zWgAAGxH6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQ6cHqAXg2Y4zzJI9Wz8Hd2XSPL+acj1cPATy/Ta9h3HNHDN2L1QOc2vXF42zxGCtcJnmyegju1NkY43yj2L1cPcAC212zs+ead7Xb89ThzvaYc66e4UbGGK8MPOccK2c5lafXvJmt7vbtfDdkx8fyLmtmH5tew3Z7njrcNUzoHsCOa6bfjud6xzUDPY54DfNhNAAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqPVg9ADczxpirZzihyyQPVw9xQhdzzserhwB4LcYY50kerZ7jxHZ7njqcI4buxeoBFrhM8mKSF1YPArfocvUAC2x3/bqOnyTJLi/kdlwzW7lM8mT1EM9qzHmsG4RP39Gcc46Vs5zKZndxn7bbK+Wt7uju+Fje0Y77vOOaE3d0d3OUsy10D2DHNdPPud7Djvu845rZwxHPtg+jAQBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUOnB6gG4mTHGXD3DCV3MOR+vHgJu0xjjPMmj1XOssNn1azubnu3LJA9XD8GrO2LoXqweYIGLJGfZ78F0NsY43yF2r58gkiQ7rPfa5eoB4A59NsmnVg/BnXtx9QALXCZ5snqIZzXmPNYL7KfvCMw5x8pZTmnTV8pJ9tjnHc/1jmtO9n4s78jZptVRzrbQ5V7abZ93W2+y55p3tOM+77jmHe24z0dcsw+jAQBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFDpweoBeDZjjPMkj1bPscIYY66eAW7Lzo/lHW12/bpM8nD1EPC0I4buxeoBTu36ifFs8RgrfDbJC6uHOKHLJE9WD3FCl6sH4GR2O9sXubpmi75+O57tQxlzHuvF5tOvjuecY+Usp7LZHYGn7XZ34GLO+Xj1EKey42M52faO7lZnO9l2n3e7Ziebne0jXreF7gHsuGb6OdcAx3LE67YPowEAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFDpweoB4NWMMc6TPFo9xwldzDkfrx4CbtuGj+XE4xnuhSOG7sXqARa4XD0AJ3E2xjjf6Mlxu3N9HXxJkl32+XrNZ4vHWGGrx/PGZzvJPmvOARtszDlXz3AjY4xXBp5zjpWznMqOa062vQu0zR7veK53X/OOdtxna+51xDUf8Y4um9joFfL2McAejvLE+Lw8nuH+8GE0AAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACo9WD0ANzPGmKtnOKHLJA9XDwF3YbPHMhtxtrlPjhi6F6sHWOAiyVn2i74XVw9wYpdJnqwe4oQuVw+wwK6PZWe732WurtkvrB7kxD6b5FOrhzihwzXYmPNYL7yefqU45xwrZzmlMcZ5kker5+BOXcw5H68e4lQ8lrfibJfb/S7ujvt8lDULXe4l+9zPHtNqx7NtzdZ8X/kwGgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJUerB7geYwx5uoZTugyycPVQ6yw0T5fzDkfrx5ihY32ONl4n6HVZtewQzli6F4kOcum0QdFdn0sn40xzneJ3THG+ct/3mXNuboxsZuL1QMscJnkxSQvrB7kxC6TPFk9xLMacx7rRcjGr5q2vaO7ke3u9F1H0KPVc6ww5xyrZziFp6/Z1kyTjXskyXHO9qFD9yj/k7k5+0yjHc+1Ne+x5h3tuM9HXLMPowEAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQ6cHqAYArY4zzJI9Wz3FiF3POx6uHgLsyxpirZzihbR/P431b7fOhHDF0L1YPcGrXAZQk2egist0+b+psjHG+0bm+XD3AAruu+cUkL6wehDv0ltUDLPDyml86znP0EUN3tzteyYZr3ih8dvcwe53vh6sHWMCa6fTO1QMs8NIrfzrMdfuIoQuVdov7zX6ky6bmnGP1DLAzH0YDAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCg0phzrp7hRsYYxxqY53GR5NHqIU5ot/UCXfa7hr0lyUurh1hjzjlWz/AsjnhH9325ejDt5nL1AHAHdjzX1ryHHdfMDt6S5KrFDuFwoTvnfG+S88VjrPBk9QBwB56sHmCBJ6sHWODJ6gEWeLJ6AE7gpQ1vvL2U91232CEc7q0LAADwLA53RxcAAJ6F0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKDS/wLBInAU1wXCaQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 231,
       "width": 349
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Maze of size (10, 10)\n"
     ]
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "%config InlineBackend.figure_format = 'retina'\n",
    "\n",
    "\"\"\" 创建迷宫并展示 \"\"\"\n",
    "maze = Maze(maze_size=10) # 随机生成迷宫\n",
    "print(maze)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1.2 重要的成员方法\n",
    "在迷宫中已经初始化一个机器人，你要编写的算法实现在给定条件下控制机器人移动至目标点。\n",
    "\n",
    "Maze 类中重要的成员方法如下：\n",
    "\n",
    "1. sense_robot() ：获取机器人在迷宫中目前的位置。\n",
    "\n",
    "> return：机器人在迷宫中目前的位置。\n",
    "\n",
    "2. move_robot(direction) ：根据输入方向移动默认机器人，若方向不合法则返回错误信息。\n",
    "\n",
    "> direction：移动方向, 如:\"u\", 合法值为： ['u', 'r', 'd', 'l']\n",
    "\n",
    "> return：执行动作的奖励值\n",
    "\n",
    "3. can_move_actions(position)：获取当前机器人可以移动的方向\n",
    "\n",
    "> position：迷宫中任一处的坐标点 \n",
    "\n",
    "> return：该点可执行的动作，如：['u','r','d']\n",
    "\n",
    "4. is_hit_wall(self, location, direction)：判断该移动方向是否撞墙\n",
    "\n",
    "> location, direction：当前位置和要移动的方向，如(0,0) , \"u\"\n",
    "\n",
    "> return：True(撞墙) / False(不撞墙)\n",
    "\n",
    "5. draw_maze()：画出当前的迷宫\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**随机移动机器人，并记录下获得的奖励，展示出机器人最后的位置。**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the history of rewards: [-0.1, -0.1, -0.1, -0.1, -0.1, -0.1, -0.1, -0.1, -0.1, -0.1]\n",
      "the actions ['d', 'u', 'd', 'r', 'l', 'r', 'u', 'r', 'l', 'd']\n",
      "the end position of robot: (1, 1)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAroAAAHPCAYAAAC8+nn2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAABYlAAAWJQFJUiTwAAASpUlEQVR4nO3dTYil2V3H8d9xOhqT1s7CF4JgEkEwiloujWI3SMCXSFCyFgRRUIIbcSUmI6KCuDG6SRYujCtBEhFcJGhVRERQaUTwZZE4JEYXUSwcSUwyHBdVE1pxSL/VPf38zucDzTTDTPX/P+e5937vM3W7x5wzAADQ5ktWDwAAADdB6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFDp1uoBHscY491J3rV6jhO7SHJ39RAnttvOu+2b2HkXdt6Dnffw/Jzz3auHeFiHu6N7Hbn3Fo+xwtnqAbhxZ6sH4CTOVg+wwNnqATiJs9UDcBL3rlvsEA4Xurm6k7vbu6ckubN6AG6cM97Djue84847cs57uJsD/V/1Q37rwsvmnGP1DKcwxpgv/3yXnXfjjPew4znvuPOOnPMeHjznozjiHV0AAPiihC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlW6tHgD438YYc/UMJ3Qx57y3eghOw7VNmzHGeZK7q+fglR0xdC9WD7DAdjtfP3kkSTZ5sbhIcpbkzuI5uFmXqwdY4DLJ7STPrR7kxM7GGOebPH8l+75OnS0eY4XLJPdXD/GwxpzHeoP94B2BOedYOQs3Z8dz3vTOwFZ3vTa9ro/1IvOU7XLOO3JtH+PaFrr8/8YYuboDM5J8Pie+UJwzjXa8ru28x8472vGcj7jzEb91gadljC9L8pYk35bkG5K86frHG5O8NleR+/I/+9kkn0jysSQfvf7rPyS5yJz/dsqxAQAehju6uxnjW5J8X5K3JvmeJF/+hF9xJvmrJB9O8qEkH8mcn3/Cr+mcqbTjdW3nPXbe0Y7nfMSdhe4OxnhVkh9J8jNJvvOGf7WPJ/mtJO/LnP/+uF/EOdNox+vaznvsvKMdz/mIOwvdZmO8Jldx+9NJvu7Ev/qnk7w/ya9kzo896r/snGm043Vt5z123tGO53zEnYVuqzF+KMl7krxh8SSfSfJLSX4tc372Yf8l50yjHa9rO++x8452POcj7uxPRmszxldmjN9O8gdZH7lJ8upche6fZ4xvXj0MALAPd3SbjPE1ufpQ2LeuHuUVvJjkBzPnR77YP+icabTjdW3nPXbe0Y7nfMSd3dFtMcbrk5zn2Y3c5OpPR/qjjPG9qwcBAPoJ3QZjfG2u/vjFN68e5SG8Jskfil0A4KYJ3Q6/meQbVw/xCF6d5HcyxutWDwIA9BK6RzfG25O8Y/UYj+H1SX519RAAQC8fRjuyMW4n+fuc/vfIfZq+O3P+2f/9m86ZRjte13beY+cd7XjOR9zZHd1j+4EcO3KT5MdXDwAAdBK6x/b21QM8BW/LGM+tHgIA6CN0j2qMV+Xqju7RfVWSt6weAgDoI3SP641JXrd4hqflO1YPAAD0EbrHdXv1AE/RV6weAADoI3SP63OrB3iKmnYBAJ4RQve4PrV6gKeoaRcA4BkhdI9qzn9N8rerx3hKPrR6AACgj9A9tg+uHuAp+OvM+fHVQwAAfYTusX1g9QBPwQdWDwAAdBK6RzbnXyb5/dVjPIF/SfKe1UMAAJ2E7vG9M8nl6iEe0zsz53+sHgIA6CR0j27OTyb5udVjPIYP5Nh3owGAZ5zQ7fC+JL+8eohH8KdJfjRzztWDAAC9hG6Dq2D8+STvWj3KQ/jjJN+fOf9z9SAAQDeh22LOmTl/McnPJnlp9Tiv4INJ3pY5/2v1IABAP6HbZs5fT/JdSf5x9SgPeDHJTyb54cz56dXDAAB7ELqN5vyLJN+e5BeSfGbxNL+X5M2Z872+JxcAOKVxtPYYY3xh4DnnWDnLIYzx1bm6m/pTSV5/ol/1M0l+N8lvZM6/eZwv4JxptON1bec9dt7Rjud8xJ2F7i7G+NIk70jyE7n61oZbN/Cr/F2S9yd5b+b81JN8IedMox2vazvvsfOOdjznI+4sdHc0xu0kd5O89frHN+Xxvo3lk0n+JMmHknw4c/7z0xvROdNnx+vaznvsvKMdz/mIOwtdXr7b+/VJ3nT9441JXpvkS5OMJJ9L8t9JPpHkY0k+muSfbvK3CHPONNrxurbzHjvvaMdzPuLOQpdnknOm0Y7XtZ332HlHO57zEXf2uy4AAFDpJj6QxA0YY5zn6vtqKbXpGV/MOe+tHgJ4cps+h/GMO2LoXqwe4NSunzzOFo+xwmWS+6uH4EadjTHON4rdy9UDLLDdc3b23HlXu71OHe7a9j26B/DgzpvZ6m7fzndDdnws77Iz+9j0OWy316nDPYcJ3QPYcWf67Xhd77gz0OOIz2E+jAYAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBApVurB+DRjDHm6hlO6DLJndVDnNDFnPPe6iEAHscY4zzJ3dVznNhur1OHc8TQvVg9wAKXSW4neW71IPAUXa4eYIHtnr+u4ydJsssbuR13ZiuXSe6vHuJhjTmPdYPwwTuac86xcpZT2ewu7oN2e6e81R3dHR/LO9rxnHfcOXFHdzdHubaF7gHsuDP9XNd72PGcd9yZPRzx2vZhNAAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACrdWj0Aj2aMMVfPcEIXc857q4eAp2mMcZ7k7uo5Vtjs+Ws7m17bl0nurB6CV3bE0L1YPcACF0nOst+D6WyMcb5D7F6/QCRJdtj32uXqAeAGvZTkxdVDcONurx5ggcsk91cP8bDGnMd6g/3gHYE551g5yylt+k45yR7nvON1vePOyd6P5R25tml1lGtb6PJM2u2cd9s32XPnHe14zjvuvKMdz/mIO/swGgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVbq0egIczxjhPcnf1HCuMMebqGeBp2fmxvKPNnr8uk9xZPQQ86Iihe7F6gFO7fmE8WzzGCi8leW71ECd0meT+6iFO6HL1AJzMbtf2Ra6es0Vfvx2v7UMZcx7rzeaD747nnGPlLKey2R2BB+12d+Biznlv9RCnsuNjOdn2ju5W13ay7Tnv9pydbHZtH/F5W+gewI470891DXAsR3ze9mE0AAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKt1aPQC8kjHGeZK7q+c4oYs5573VQ8DTtuFjOfF4hmfCEUP3YvUAC1yuHoCTOBtjnG/04rjddX0dfEmSXc75euezxWOssNXjeeNrO8k+O+eADTbmnKtneCRjjC8MPOccK2c5lR13Tra9C7TNGe94Xe++8452PGc79zrizke8o8smNnqHvH0MsIejvDA+KY9neHb4MBoAAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlW6tHoBHM8aYq2c4ocskd1YPATdhs8cyG3Ft8yw5YuherB5ggYskZ9kv+m6vHuDELpPcXz3ECV2uHmCBXR/Lru1+l7l6zn5u9SAn9lKSF1cPcUKHa7Ax57HeeD34TnHOOVbOckpjjPMkd1fPwY26mHPeWz3EqXgsb8W1XW73u7g7nvNRdha6PJOccz9nTKsdr2072/lZ5cNoAABUEroAAFQSugAAVBK6AABUEroAAFQSugAAVBK6AABUEroAAFQSugAAVBK6AABUEroAAFQSugAAVBK6AABUEroAAFQSugAAVBK6AABUEroAAFQSugAAVBK6AABUEroAAFQSugAAVBK6AABUEroAAFQSugAAVBK6AABUEroAAFQSugAAVBK6AABUEroAAFQSugAAVBK6AABUEroAAFQSugAAVBK6AABUEroAAFQSugAAVLq1eoAnMcaYq2c4ocskd1YPscJG53wx57y3eogVNjrjZONzhlabPYcdyhFD9yLJWTaNPiiy62P5bIxxvkvsjjHOX/75Ljvn6sbEbi5WD7DAZZLbSZ5bPciJXSa5v3qIhzXmPNabkI3fNW17R3cj293pu46gu6vnWGHOOVbPcAoPPmfbmSYb90iS41zbhw7do/xH5tE5ZxrteF3beY+dd7TjOR9xZx9GAwCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKDSrdUDAFfGGOdJ7q6e48Qu5pz3Vg8BN2WMMVfPcELbPp7H81ud86EcMXQvVg9watcBlCTZ6Elku3Pe1NkY43yj6/py9QAL7Lrz7STPrR6EG/SG1QMs8PLOLxznNfqIobvbHa9kw503Cp/d3cle1/ed1QMsYGc6/djqARZ44Qs/O8zz9hFDFyrtFveb/S9dNjXnHKtngJ35MBoAAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVxpxz9QyPZIxxrIF5EhdJ7q4e4oR22xfost9z2BuSvLB6iDXmnGP1DA/jiHd0n8/Vg2k3l6sHgBuw43Vt5z3suDM7eEOSqxY7hMOF7pzz3UnOF4+xwv3VA8ANuL96gAXurx5ggfurB1jg/uoBOIEXNrzx9kKev26xQzjcty4AAMDDONwdXQAAeBhCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEr/A2qq6s5xqxvoAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 231,
       "width": 349
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Maze of size (10, 10)\n"
     ]
    }
   ],
   "source": [
    "import random\n",
    "\n",
    "rewards = [] # 记录每走一步的奖励值\n",
    "actions = [] # 记录每走一步的移动方向\n",
    "\n",
    "# 循环、随机移动机器人10次，记录下奖励\n",
    "for i in range(10):\n",
    "    valid_actions = maze.can_move_actions(maze.sense_robot())\n",
    "    action = random.choice(valid_actions)\n",
    "    rewards.append(maze.move_robot(action))\n",
    "    actions.append(action)\n",
    "\n",
    "print(\"the history of rewards:\", rewards)\n",
    "print(\"the actions\", actions)\n",
    "\n",
    "# 输出机器人最后的位置\n",
    "print(\"the end position of robot:\", maze.sense_robot())\n",
    "\n",
    "# 打印迷宫，观察机器人位置\n",
    "print(maze)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "toc-hr-collapsed": false
   },
   "source": [
    "## 2.2 基础搜索算法介绍（广度优先搜索算法）\n",
    "\n",
    "对于迷宫游戏，常见的三种的搜索算法有广度优先搜索、深度优先搜索和最佳优先搜索（A*)。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "toc-hr-collapsed": false
   },
   "source": [
    "在下面的代码示例中，将实现广度优先搜索算法；主要通过建立一颗搜索树并进行层次遍历实现。\n",
    "+ 每个节点表示为以 `Class SearchTree` 实例化的对象，类属性有：**当前节点位置、到达当前节点的动作、当前节点的父节点、当前节点的子节点**；\n",
    "+ `valid_actions():` 用以获取机器人可以行走的位置（即不能穿墙）；\n",
    "+ `expand():` 对于未拓展的子节点进行拓展；\n",
    "+ `backpropagation():` 回溯搜索路径。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2.1 算法具体步骤\n",
    "\n",
    "首先以机器人起始位置建立根节点，并入队；接下来不断重复以下步骤直到判定条件:\n",
    "\n",
    "1. 将队首节点的位置标记已访问；判断队首是否为目标位置(出口)， **是** 则终止循环并记录回溯路径\n",
    "2. 判断队首节点是否为叶子节点，**是** 则拓展该叶子节点\n",
    "3. 如果队首节点有子节点，则将每个子节点插到队尾\n",
    "4. 将队首节点出队"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2.2 编程实现广度优先搜索算法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "# 机器人移动方向\n",
    "move_map = {\n",
    "    'u': (-1, 0), # up\n",
    "    'r': (0, +1), # right\n",
    "    'd': (+1, 0), # down\n",
    "    'l': (0, -1), # left\n",
    "}\n",
    "\n",
    "\n",
    "# 迷宫路径搜索树\n",
    "class SearchTree(object):\n",
    "\n",
    "\n",
    "    def __init__(self, loc=(), action='', parent=None):\n",
    "        \"\"\"\n",
    "        初始化搜索树节点对象\n",
    "        :param loc: 新节点的机器人所处位置\n",
    "        :param action: 新节点的对应的移动方向\n",
    "        :param parent: 新节点的父辈节点\n",
    "        \"\"\"\n",
    "\n",
    "        self.loc = loc  # 当前节点位置\n",
    "        self.to_this_action = action  # 到达当前节点的动作\n",
    "        self.parent = parent  # 当前节点的父节点\n",
    "        self.children = []  # 当前节点的子节点\n",
    "\n",
    "    def add_child(self, child):\n",
    "        \"\"\"\n",
    "        添加子节点\n",
    "        :param child:待添加的子节点\n",
    "        \"\"\"\n",
    "        self.children.append(child)\n",
    "\n",
    "    def is_leaf(self):\n",
    "        \"\"\"\n",
    "        判断当前节点是否是叶子节点\n",
    "        \"\"\"\n",
    "        return len(self.children) == 0\n",
    "\n",
    "\n",
    "def expand(maze, is_visit_m, node):\n",
    "    \"\"\"\n",
    "    拓展叶子节点，即为当前的叶子节点添加执行合法动作后到达的子节点\n",
    "    :param maze: 迷宫对象\n",
    "    :param is_visit_m: 记录迷宫每个位置是否访问的矩阵\n",
    "    :param node: 待拓展的叶子节点\n",
    "    \"\"\"\n",
    "    can_move = maze.can_move_actions(node.loc)\n",
    "    for a in can_move:\n",
    "        new_loc = tuple(node.loc[i] + move_map[a][i] for i in range(2))\n",
    "        if not is_visit_m[new_loc]:\n",
    "            child = SearchTree(loc=new_loc, action=a, parent=node)\n",
    "            node.add_child(child)\n",
    "\n",
    "\n",
    "def back_propagation(node):\n",
    "    \"\"\"\n",
    "    回溯并记录节点路径\n",
    "    :param node: 待回溯节点\n",
    "    :return: 回溯路径\n",
    "    \"\"\"\n",
    "    path = []\n",
    "    while node.parent is not None:\n",
    "        path.insert(0, node.to_this_action)\n",
    "        node = node.parent\n",
    "    return path\n",
    "\n",
    "\n",
    "def breadth_first_search(maze):\n",
    "    \"\"\"\n",
    "    对迷宫进行广度优先搜索\n",
    "    :param maze: 待搜索的maze对象\n",
    "    \"\"\"\n",
    "    start = maze.sense_robot()\n",
    "    root = SearchTree(loc=start)\n",
    "    queue = [root]  # 节点队列，用于层次遍历\n",
    "    h, w, _ = maze.maze_data.shape\n",
    "    is_visit_m = np.zeros((h, w), dtype=np.int32)  # 标记迷宫的各个位置是否被访问过\n",
    "    path = []  # 记录路径\n",
    "    while True:\n",
    "        current_node = queue[0]\n",
    "        is_visit_m[current_node.loc] = 1  # 标记当前节点位置已访问\n",
    "\n",
    "        if current_node.loc == maze.destination:  # 到达目标点\n",
    "            path = back_propagation(current_node)\n",
    "            break\n",
    "\n",
    "        if current_node.is_leaf():\n",
    "            expand(maze, is_visit_m, current_node)\n",
    "\n",
    "        # 入队\n",
    "        for child in current_node.children:\n",
    "            queue.append(child)\n",
    "\n",
    "        # 出队\n",
    "        queue.pop(0)\n",
    "\n",
    "    return path\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**测试广度优先搜索算法**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "搜索出的路径： ['d', 'r', 'd', 'r', 'u', 'r', 'r', 'r', 'r', 'u', 'r', 'd', 'd', 'l', 'd', 'r', 'd', 'r', 'd', 'd', 'd', 'r', 'd', 'd']\n",
      "恭喜你，到达了目标点\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAroAAAHPCAYAAAC8+nn2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAABYlAAAWJQFJUiTwAAATXElEQVR4nO3dXYymZ13H8d+9O22NLd0tIKmIlhajFdJkTdHgS5whEU00ygECRjzwBTUkxnBi9EDjNsbEhHhAQqIRTdSYmBANAT1QIDKTWESIMvJWUdmyoUVo1HZgCX1h9/JgZtpBrN12nue+5v5fn0+y6bQHs/+r1/Xc8537eeaZqbUWAACo5lTvAQAAYB2ELgAAJQldAABKEroAAJQkdAEAKEnoAgBQktAFAKAkoQsAQElCFwCAkoQuAAAlCV0AAEoSugAAlCR0AQAoSegCAFCS0AUAoCShCwBASUIXAICShC4AACUJXQAAShK6AACUJHQBAChJ6AIAUJLQBQCgJKELAEBJQhcAgJKELgAAJQldAABKEroAAJQkdAEAKEnoAgBQktAFAKCkjd4DPBPTNJ1P8hu955jZTpLN3kPMbLQ1j7bexJpHYc1jsOYx3NVaO997iKu1uDu6B5G71XkMWIdzvQeANTnXewBgZbYOWmwRFhe62b+TO9p3T4zhTO8BYE2cbahjMwt6Vn2RL1041Fqbes8AqzBNUzv82LmmEmcb6jj6eF6KJd7RBQCApyR0AQAoSegCAFCS0AUAoCShCwBASUIXAICShC4AACUJXQAAShK6AACUJHQBAChJ6AIAUJLQBQCgJKELAEBJQhcAgJKELgAAJQldAABKEroAAJQkdAEAKEnoAgBQktAFAKAkoQsAQElCFwCAkoQuAAAlCV0AAEoSugAAlCR0AQAoSegCAFCS0AUAoCShCwBASUIXAICShC4AACUJXQAAShK6AACUJHQBAChJ6AIAUJLQBQCgpI3eA3B1pmnaTrLZe46Z7SU503uIuU3T1HrPMKOd1tpW7yHmNOhjmQEMeraHu4YtzRJDd6f3AMzmht4DzOxyktO9h4A12Euy23uIuRwEX5JEBJV3bpqm7YH2eXENtsTQHe27xZGNFn2XMuAdbIawO1AIJL5OjeRMxtrvxa11am1Zz5IefVq3tTb1nIX1sc/AUrl+jWHEfV7imv0wGgAAJQldAABKEroAAJQkdAEAKEnoAgBQktAFAKAkoQsAQElCFwCAkoQuAAAlCV0AAEoSugAAlCR0AQAoSegCAFCS0AUAoCShCwBASUIXAICShC4AACUJXQAAShK6AACUJHQBAChJ6AIAUJLQBQCgJKELAEBJQhcAgJKELgAAJQldAABKEroAAJQkdAEAKEnoAgBQktAFAKAkoQsAQElCFwCAkoQuAAAlCV0AAEoSugAAlCR0AQAoaaP3AMC+aZq2k2z2nmNmO621rd5DsF6Dnu3hjLzP0zS13jPwf1ti6O70HmBuBxePJMlAUTDUPh/s8bnOY7BmIz6WBz7be0l2ew8Ba7Cosz21tqxvQo5+19Ram3rOMpcR1zyage8GDHVHd8THsrM9jpHv6I5oKdcwobsAI655NPZ4DCPu84hrZgwjnu0lrtkPowEAUJLQBQCgJKELAEBJQhcAgJKELgAAJQldAABKEroAAJQkdAEAKEnoAgBQktAFAKAkoQsAQElCFwCAkoQuAAAlCV0AAEoSugAAlCR0AQAoSegCAFCS0AUAoCShCwBASUIXAICShC4AACUJXQAAShK6AACUJHQBAChJ6AIAUJLQBQCgJKELAEBJQhcAgJKELgAAJQldAABKEroAAJQkdAEAKEnoAgBQktAFAKAkoQsAQEkbvQc4jmmaWu8Z5jbYmndaa1u9h5jbYHu8l+RM7yF6GGyfhzNN03aSzd5zzGzYxzMn1xJDdyfJuYz3YLqc5HTvIVibUc81Y9hLstt7CFiD0c72Tu8Bnq4lhu5o3yEfuhQRVFZrbWvQO0CMYXfEZ2cYwmhne3Ffo6bWlvXs2dGn+1prU89ZAJ4O1y9gyZZ4DfPDaAAAlCR0AQAoSegCAFCS0AUAoCShCwBASUIXAICShC4AACUJXQAAShK6AACUJHQBAChJ6AIAUJLQBQCgJKELAEBJQhcAgJKELgAAJQldAABKEroAAJQkdAEAKEnoAgBQktAFAKAkoQsAQElCFwCAkoQuAAAlCV0AAEoSugAAlCR0AQAoSegCAFCS0AUAoCShCwBASUIXAICShC4AACUJXQAAShK6AACUJHQBAChJ6AIAUJLQBQCgpI3eA3B1pmnaTrLZe46Z7bTWtnoPMRd7DCzZoNewvSRneg/Bk1ti6O70HmBuBxePc53HgHU4N03T9kCxu9d7gLkdXL+SJKPs84hrZih7SXZ7D3G1lhi6o323mIy5ZsZwJmOd7xHv/Iy0v4dGXDPjWNR1e2qt9Z7haZmm6fGBW2tTz1nmMuKaqW/Ec23N1gxLtsSz7YfRAAAoSegCAFCS0AUAoCShCwBASUIXAICShC4AACUJXQAAShK6AACUJHQBAChJ6AIAUJLQBQCgJKELAEBJQhcAgJKELgAAJQldAABKEroAAJQkdAEAKEnoAgBQktAFAKAkoQsAQElCFwCAkoQuAAAlCV0AAEoSugAAlCR0AQAoSegCAFCS0AUAoCShCwBASUIXAICShC4AACUJXQAAShK6AACUJHQBAChJ6AIAUJLQBQCgpI3eAxzHNE2t9wxzG2zNO621rd5DsH6Dneth2WeqmaZpO8lm7zl4cksM3Z0k55Kc6TzH3C4nOd17CNbj4GKZJBko7veS3JDxzvVekt3eQ8xo5Gv2pd5DzGXQa9ioFnUNm1pb1jfYA98R2Mt4XyiGuaN79Fy31qaes8xl4MfyMOf60Mh3vUZ8PI+y5sTZ7j3D1Vh06C7lfzI8lRHP9YhrZgwjnu0R1zyiJe6zH0YDAKAkoQsAQElCFwCAkoQuAAAlCV0AAEoSugAAlCR0AQAoSegCAFCS0AUAoCShCwBASUIXAICShC4AACUJXQAAShK6AACUJHQBAChJ6AIAUJLQBQCgJKELAEBJQhcAgJKELgAAJQldAABKEroAAJQkdAEAKEnoAgBQktAFAKAkoQsAQElCFwCAkoQuAAAlCV0AAEoSugAAlCR0AQAoSegCAFCS0AUAoCShCwBASUIXAICShC4AACVt9B7gOKZpar1nmNFekjO9h5jZTmttq/cQcxvsXCcZbs0jPpZHXHOS4c72cKZp2k6y2XsOntwS7+juZP+iCZWMeq4v9x6ggxt6D8AsRjzbe9m/llHbovZ5iaG7mUHvDFDXwZ3r3c5j9HCp9wAdnO49ALMY8Wzvjvgs3IDOZEF3safWlvWsytGngVprU89ZAJ4O1y9gyZZ4DVviHV0AAHhKQhcAgJKELgAAJQldAABKEroAAJQkdAEAKEnoAgBQktAFAKAkoQsAQElCFwCAkoQuAAAlCV0AAEoSugAAlCR0AQAoSegCAFCS0AUAoCShCwBASUIXAICShC4AACUJXQAAShK6AACUJHQBAChJ6AIAUJLQBQCgJKELAEBJQhcAgJKELgAAJQldAABKEroAAJQkdAEAKEnoAgBQktAFAKAkoQsAQElCFwCAkoQuAAAlbfQeABjXNE3bSTZ7z8F6DbrPO621rd5DsF6Dnu1FWWLo7vQeYG4HD6QkySgXztHWPNp6B3c5yaXeQ8zl4Gyf6zwGa+YaNpS9JLu9h7haU2ut9wxPyzRNjw/cWpt6zjIXa66/5tHWe2jkuyGj7PPRsz2Yoe7ouoaNZyn7LHQXwJrrr3m09Y5qxH0ecc0jss9jWOI++2E0AABKEroAAJQkdAEAKEnoAgBQktAFAKAkoQsAQElCFwCAkoQuAAAlCV0AAEoSugAAlCR0AQAoSegCAFCS0AUAoCShCwBASUIXAICShC4AACUJXQAAShK6AACUJHQBAChJ6AIAUJLQBQCgJKELAEBJQhcAgJKELgAAJQldAABKEroAAJQkdAEAKEnoAgBQktAFAKAkoQsAQElCFwCAkoQuAAAlCV0AAEoSugAAlCR0AQAoSegCAFDSRu8BeHqmaWq9Z2C9BtvjndbaVu8hehhsnwG6WGLo7vQeoIOdJOeSnOk8x9wuJ7nUe4iZ7CW5Icnp3oOwVqPu816S3d5DzGWapu3Djwf6Rm6v9wBzG3SfF9dgU2vLuqlw9C5Ia23qOcucDh5Qm73n6GGEfR747t5Qd3Tt8xhG/DplzdZ8UgldTqTR9nm09Y7KPo9hxH22Zms+qfwwGgAAJQldAABKEroAAJQkdAEAKEnoAgBQktAFAKAkoQsAQElCFwCAkoQuAAAlCV0AAEoSugAAlCR0AQAoSegCAFCS0AUAoCShCwBASUIXAICShC4AACUJXQAAShK6AACUJHQBAChJ6AIAUJLQBQCgJKELAEBJQhcAgJKELgAAJQldAABKEroAAJQkdAEAKEnoAgBQktAFAKAkoQsAQElCFwCAkoQuAAAlCV0AAEoSugAAlLTRewDgK03T1HrPMKO9JGd6D9HDYPu801rb6j1ED4Pt87Cmu+zzSbXE0N3pPcDcpmnaPvx4oC8We70HmNlOknMZNPqgmL0kNyQ53XuQme0l2e09xGyu6z1AB7cc/PPiclpsiaG72XuADkZc81DB11rbOviGZsS9hmqGun4dsTvQzZjkkd4DdHDx8Y8W87VqiaELJQ31BWJQR5/Gbq1NPWdhHvYZ+hK6AAAnyDVfTl70YHLrg8ltDya3PpTc8lByw6PJNVeSUy159HTy8EZy343JvWeTe29KLtyU/Puzk0sjvqziSQhdAICeWnLH55Lvv5C84kLyfReT6x97Zp/q8pR88PnJe25L3v2i5O9fkDw2cO1NrS3rBwVHfOrPmsdYM/U512Owz2P4infUOP/MPsfNX0je8MHk9f+UPP/SauZqSY4eukvXJG97SfLmlyUfvvmYn/z8kb9nIWdb6C6ANY+xZupzrsdgn8dwnNC98/7kje9PXvOx5Norq53r/7N9y37wvvNbkyvP5DcpnH/iw6Wc7YFvZgMAzOfmLyS/8zfJT3x0/9/nvNXYkmxd3P/zwecnb/jh5B+/YcYBOvGb0QAA1qklr/vn5J637EfuYeDOeUv06N/1HZ9JPvDW5LffnVz75RmH6EDoAgCsyakryVvfmfzp25OzB++9exKe85+S/Mrdyd1/mDzni72nWR+hCwCwBqcvJ3/y9uT1H5r3ZQpX4zC2X/ofyXv/OHnein4Y7qQRugAAK7ZxOfmzv0he95GvfieEk+aOB5LtP0q+/vO9J1k9oQsAsGK/vpO8+uMnP3IPfdt/7of5NOO7QMxB6AIArNCLH0h+9e/2P15C5Cb7Qb55MfnZD/WeZLWELgDAipy6kvzBO+d9f9xVOAzyN71r/23QqhC6AAAr8vJ7k++6r/cUz9zZR5Jf/EDvKVZH6AIArMgrP9F7guP70QJrOCR0AQBWoSWv/JfeQxzfHQ8kt/537ylWQ+gCAKzACx9KvqnIW3Rtfar3BKshdAEAVuCmL/WeYHXOPtx7gtUQugAAK3DqpP36s2OoshahCwCwAp+/rvcEq1NlLUIXAGAFPvns5IGv7T3FarzvG3tPsBpCFwBgBa6cSv7qW3pPcXwXziYfe17vKVZD6AIArMg7bu89wfG94/Ys53cXPwWhCwCwIn/9zck9z+09xTP38Onkd1/ae4rVEboAACvy6Ebycz+y//ES37jgNzeTf1twqP9vQhcAYIXuviX5vTuX9+z/R56XvOm7e0+xWkIXAGDFfvkHkrsP3rlgCXd2P3t98tofSx7b6D3JagldAIAVu3Rd8oM/mWzfsn9n9yTH7v3PSjZ/OrmnyDstHCV0AQDW4IvXJT/0uuRdt528lzEchvfFM8nmTyX/Wuh1uUcJXQCANfnStfux+2svTx47QdU1JXnbi5Nv/4Xkk8/pPc36nKD/5QAA9Vw+nfzWZvLSn0/e94In/nuvlzN8+sbkVa9JXvvq5MEiv8ntyRR7yTEAwMn04ZuT7/2Z5FX3JG98f/I9n97/7y3re2nD0c994Wzylu9Mfv/O/ZdVjEDoAgDMpJ1K/vwl+3/uvD/5pX9IfvyjybVX1vP3TUn+9oXJm1+2/+uJrwz2XP7U2kn+OcCvNk3T4wO31k7aa7vXwprHWDP1OddjsM9jOLrPOX+8z/Wsh5OtTyWvuJC84pPJ7f91vM/32euT99yWvPtF+//8zI3H+3yPO//Eh0s52+7oAgB09IWvSf7y9v0/SfKCveSOzyW3PpTc+uD+P1/4UHL9o8k1V5JTLXn0dPLI6eS+G5N7b0ou3JTcezb5xHOTj39dTt7bPHQidAEATpD7zuz/4fgGe6UGAACjWPRrdKGYnSSbvYeY2YhrHpF9HoN9HshSXqO7xDu6d2X/wTSavd4DdDDimqnPuR6DfR7BIG/R9RVuSbLfYouwuNBtrZ1Pst15jB52ew/QwW7vAWANdnsPwCx2ew/ADB4Z8Mbbxdx10GKLsLiXLgAAwNVY3B1dAAC4GkIXAICShC4AACUJXQAAShK6AACUJHQBAChJ6AIAUJLQBQCgJKELAEBJQhcAgJKELgAAJQldAABKEroAAJQkdAEAKEnoAgBQktAFAKAkoQsAQElCFwCAkoQuAAAlCV0AAEoSugAAlCR0AQAoSegCAFCS0AUAoCShCwBASUIXAICShC4AACUJXQAAShK6AACUJHQBAChJ6AIAUNL/AL9z58Pcj9zJAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 231,
       "width": 349
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Maze of size (10, 10)\n"
     ]
    }
   ],
   "source": [
    "maze = Maze(maze_size=10)\n",
    "height, width, _ = maze.maze_data.shape\n",
    "\n",
    "path_1 = breadth_first_search(maze)\n",
    "print(\"搜索出的路径：\", path_1)\n",
    "\n",
    "for action in path_1:\n",
    "    maze.move_robot(action)\n",
    "\n",
    "if maze.sense_robot() == maze.destination:\n",
    "    print(\"恭喜你，到达了目标点\")\n",
    "\n",
    "print(maze)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "toc-hr-collapsed": false
   },
   "source": [
    "## 2.3 题目一: 实现基础搜索算法(总分40分)\n",
    "* 题目要求： 任选深度优先搜索算法、最佳优先搜索 A* 算法其中一种实现机器人走迷宫\n",
    "\n",
    "* 输入：迷宫\n",
    "\n",
    "* 输出：到达目标点的路径\n",
    "\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3.1 编写您的基础搜索算法\n",
    "深度优先\n",
    "1. 对当前节点递归扩展子节点，若没有后续子节点则返回上一层扩展其他子节点\n",
    "2. 若没有未搜索的子节点，则对该节点进行标记"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_new_pos(pos, direction):\n",
    "    \"\"\"\n",
    "    返回执行移动后的坐标 (x, y)\n",
    "    \"\"\"\n",
    "    if direction == 'u':\n",
    "        return (pos[0] - 1, pos[0])\n",
    "    if direction == 'd':\n",
    "        return (pos[0] + 1, pos[0])\n",
    "    if direction == 'l':\n",
    "        return (pos[0], pos[0] - 1)\n",
    "    if direction == 'r':\n",
    "        return (pos[0], pos[0] + 1)\n",
    "\n",
    "def depth_first_search(maze, pos, is_visit_m):\n",
    "    is_visit_m[pos] = 1     # 标记该位置已被搜索\n",
    "\n",
    "    print(pos)\n",
    "    print(is_visit_m)\n",
    "\n",
    "    actions = maze.can_move_actions(current_pos)\n",
    "    for action in actions:\n",
    "        new_pos = get_new_pos(current_pos, action)\n",
    "\n",
    "        if is_visit_m[new_pos]:\n",
    "            # 如果当前子节点已经被搜索过，则尝试扩展另外的子节点\n",
    "            continue    # 继续进入for循环\n",
    "        else:\n",
    "            # 如果新节点没有被搜索过，则继续入栈\n",
    "            queue.append(new_pos)\n",
    "            current_pos = new_pos\n",
    "            break       # 继续进入while循环\n",
    "    return "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {
    "deletable": false,
    "select": true
   },
   "outputs": [],
   "source": [
    "def my_search(maze):\n",
    "    \"\"\"\n",
    "    任选深度优先搜索算法、最佳优先搜索（A*)算法实现其中一种\n",
    "    :param maze: 迷宫对象\n",
    "    :return :到达目标点的路径 如：[\"u\",\"u\",\"r\",...]\n",
    "    \"\"\"\n",
    "\n",
    "    path = []\n",
    "    \n",
    "    # -----------------请实现你的算法代码--------------------------------------\n",
    "    start = maze.sense_robot()\n",
    "    root = SearchTree(loc=start)\n",
    "    queue = [root]\n",
    "    h, w, _ = maze.maze_data.shape\n",
    "    is_visit_m = np.zeros((h, w), dtype=np.int32)\n",
    "\n",
    "    while True:                \n",
    "        current_node = queue[-1]            # 当前位置为栈顶元素\n",
    "\n",
    "        if current_node.loc == maze.destination:\n",
    "            path = back_propagation(current_node)\n",
    "            break\n",
    "        # print(current_node.loc)\n",
    "\n",
    "        # 如果已经搜索过，弹栈\n",
    "        if is_visit_m[current_node.loc]:\n",
    "            queue.pop()\n",
    "            continue\n",
    "\n",
    "        # 如果是叶节点则扩展子节点\n",
    "        if current_node.is_leaf():\n",
    "            expand(maze, is_visit_m, current_node)\n",
    "\n",
    "        # 如果没有子节点，弹栈\n",
    "        if not current_node.children:\n",
    "            queue.pop()\n",
    "        else:\n",
    "            for child in current_node.children:\n",
    "                queue.append(child)\n",
    "\n",
    "        is_visit_m[current_node.loc] = 1    # 标记该位置已被搜索\n",
    "        \n",
    "    # -----------------------------------------------------------------------\n",
    "    return path\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3.2 测试您编写的基础搜索算法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAroAAAHPCAYAAAC8+nn2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAABYlAAAWJQFJUiTwAAAShElEQVR4nO3dQYhl6VnG8edLd0I0rQ0Ss5MkgjsjTRRUJHaDJhEU4kIQBIWABBESQcRVxGTjwp2YjUZEERQEF2o2khi6FipZiC1EXBgSR1FBBVNmkJih87momtADkzg9VXW/Ps/3+0ExvZnq9+1z7rn/e+pW1ZhzBgAA2rxm9QAAAHAThC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVbq8e4NUYY3woyS+vnuPEzpLcXz3Eie228277JnbehZ33YOc9fHjO+aHVQ7xSh7ujexm5DxaPAQC81L3VAyxwb/UACzy4bLFDOFzo5uJO7m6vngDgWXd39QAL7Ljz/Rzoq+qHfOvCi+acY/UMALC7McZ88c+7PDfvvvNRHDp0b8QYI8kbk7z1iY9vTPLaJLeSvJDkS0n+NcnnLj+ey5xfXDIvAAAvS+iOcTvJdyV55+XH25O84VV8nn/KxZvSP57kE5nz365xSgAAntKY81h3oa/lSwUXd20fJPmZJO/OzbzH5tNJ/jDJb2TOf7+Bzw8Az4Tdv4xv52fXXqE7xtcl+YkkH0jyHdc72Vf1pSR/kOTXMuffnOjvBICTOWIAXZWdj7HzHqF7cQf3p5L8apI33cxkr8jHkvxc5vzswhkA4FodMYCuys7H2PmIP17s6YzxliSfTPI7WRu5SfIjSf4uY/x8xuj/twcAWKj7ju4Y70nye0m+4YbHejX+PMmPZc7Prx4EAK7iiHf6rsrOx9i5967iGD+e5I/ybEZukvxAkk9kjG9aPQgAQKPO0B3jJ5P8fi5+7u2z7DuTfDJjfPPqQQAA2vS9dWGMB7l4T+4hbqlf+qsk78icj1cPAgBP64hf0r4qOx9j5647uhc/Puw3c6zITZLvTfKzq4cAAGjSFbrJB5N82+ohXqVfyRjfsnoIAIAWPaE7xhuT/OLqMa7gTpJfWj0EAECLntBNfjjJ7dVDXNF7/HxdAIDr0RRV71k9wDV4U5LvXj0EAECDjtC9+BW/71o9xjV59+oBAAAadIRu8vVJ3rB6iGuy+tcUAwBUaAnd160e4Bo17QIAsExL6D6/eoBr1LQLAMAyHaE75wtJ/n71GNfkb1cPAADQoCN0L/zx6gGuwUzysdVDAAA0aArdP1k9wDX4y8z5H6uHAABo0BS6n0ry6dVDXNFvrR4AAKBFT+jO+eUkP52LL/8f0cMkv7t6CACAFj2hmyRzfirJr68e41X43yTvy5xHjXQAgGdOV+he+GCSv149xFN6f+b8h9VDAAA06QvdOb+Q5Adz8Z7dZ93MxZ3cj64eBACgTV/oJsmcn0/yriR/sXiSr+XLSd4rcgEAbkZn6CbJnP+di9j9yOpRXsa/JHl35vTNZwAAN6Q3dJNkzv/JnO9P8s4kz8p7YD+a5G2Z8xOrBwEAaNYdui+6iMq3JfmFJP+8YoIkf5rkezLn+zLnfy2YAQBgK+NoP9FqjPGVgeec41V8gttJfjTJB5K849oGe3lfSPLbST6SOT9zw38XACxx5efmA7LzMXbeL3Rf+sm+PckP5eKtDd+f5PVX+nwX/jHJxy8//uzyvcIAUOuIAXRVdj7GznuH7ks/8euTfF+Styd5a5JvvfzvW5K87mX+j/9M8rnLj88m+UwufrvZZ/3iBwB2csQAuio7H2Nnofv//4WvyUXovjbJrSQvJPlS5nzhxv9uADiAIwbQVdn5GDvfXj3AM2/OLyf54uUHAAAHcejQffKVxQbO5pwPVg9xSmOMh0nur57jhLY7xjva8LxOnNvAIkcM3bMk95LcXTzHqd0bYzz0ZFFtq2N8GXxJkl123phzu9/Z6gEWsPMBHPo9ujs6yntirsOmd762OcZHfK/Xddj1vE72Oc67ntv0O+K5fejQPco/8lXtuPNudjzGO+68ox2P8447s4cjntt7/GY0AAC2I3QBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqHR79QDAS40x5uoZTm2znc+T3F09xAqbHeck2+2847l9Nud8sHoIvrojhu7Z6gEW2G7nMcbDF/+8yUXkLMm97Pck8TjJrdVDnNid1QMssONx3nHnHc/te2OMh5s8TyUH7JEx57FebD756njOOVbOws3Z8Thfxv391XOc2I53gHa043HecedtbfQ8dbjnZqHLM8lxppHzmlY7ntt2PsbOvhkNAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEq3Vw9wFWOMuXqGEzpPcnf1EABPa4zxMMn91XOcmGv2RjbrkUM5YuieJbmX/S4gd1YPsMB5kkerhziFyxBIksw5H6yb5HR23DkX1y9otc01+9KuPXKo4zzmPNaLEK+atnK2SwA9eV7POcfKWU5lx5135Y7uNra5Zr9o03M7yXGu24cO3aP8I1/VjjvvZsdjvOPOAEd2xOu2b0YDAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCg0u3VA/B0xhhz9QwndDbnfLB6iFPb7BhDpTHGwyT3V89xYudJ7q4e4sR23PlQjhi6Z6sHWOA8yZ0kt1YPwo05S3Iv+10wHyd5fvUQp3IZP0mSXV7E7bgzW7mzeoAFzpM8Wj3EKzXmPNbNoyfvds05x8pZTmXjO3xb3dHd9A5Qkj0fy3butunj2d3NjRzl8Sx0D2DHnem343lt5z12Zg87nttH3Nk3owEAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQSegCAFBJ6AIAUEnoAgBQ6fbqAYALY4yHSe6vnoPTGGPM1TNwczZ9PJ8nubt6CHjSEUP3bPUAC2y38+WTRJJkzvlg3STcsMdJnl89xAmdJ7mT5NbqQU7sPMmj1UNw4+6sHmCB3a5hh+uRI4bubq+Qdw297Y7zpm5lrztAO+36pEebXsd2s9sLuGS/a9jhnpvHnMf66tmTX+6bc46Vs3BzHOd+Ox7jHXdmDzue23Y+xs6+GQ0AgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKt1cPAFwYYzxMcn/1HCuMMebqGU5ts53P5pwPVg9xSjs/nuFZcsTQPVs9wKldXjCTJBs9WZyvHgBuyOMkt1YPATfgPMmj1UOc0HY9kgPufMTQ3fEV8o473109ANyQ5+P8ptOjjW7G7HTj6UmH65Ex57G+evbkl/vmnGPlLKdi5z123o1jDHAsR7xu+2Y0AAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACrdXj0AT2eMMVfPwM0YYzxMcn/1HMDVbfp4PptzPlg9BDzpiKF7tnqABc6T3Elya/UgJ3ae5NHqIbhRj5M8v3qIU7mMnyTJLkGw484buzfGeLjLcd703D5cg405j3WD8Mk7mnPOsXKWU9n4Lu5Wdwc2vQOUZM/Hsp27eTz32/HcPuLOQvcAdtyZfjue13beY+cd7Xic7XyMnX0zGgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJVurx7gKsYYc/UMcF3GGA+T3F89xwoey3vY7DifzTkfrB5ihc2OM8+4I4buWZJ7Se4unuPUHid5fvUQp3IZfUmSHZ4sLve9t3iMFR4nubV6iBM7T/Jo9RAndJ7kTvY7zvfGGA93uH5d2vU47/Z4Pls9wNMacx7rhdfurxTnnGP1DKfw5HHeYeeNz+vz7Peidas7fRuf20n2uH4lWx/nbR/PRzm3j3hH9yuO8o98VRtfQLa0y3nNfnY5t3e/Zu9ynDkG34wGAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQKXbqwfg6Ywx5uoZuFmbHeOzOeeD1UPATdns8Zxks53fnOS9q4fgazli6J6tHmCBsyT3ktxdPMepnSd5tHqIEzlPcifJrdWDcHPGGA9f/PNGgb/jNXvXx/Pj7LfzXt58+d/njvO4PmLo3l89wKnNOR9cPkHutvujjWJgtxcxu9rtMbxT0D9p18fz89l39z0895U/HeZadsTQ3dKmTxZbmnOO1TMA18Pjudf48EZv0Tgw34wGAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBApTHnXD3DUxljHGvg63OW5P7qIU5sx5134xjvwXHew37H+c1Jnls9xBpzzrF6hlfiiHd0P5yLBxO0OV89ACfhOO/BcabTm5NctNghHC5055wfSvJw8RhwEx6tHoCTeLR6AE7i0eoBOIHnNrzx9lw+fNlih3C4ty4AAMArcbg7ugAA8EoIXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACr9H71PIixanOBwAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 231,
       "width": 349
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Maze of size (10, 10)\n",
      "搜索出的路径： ['r', 'd', 'r', 'd', 'r', 'r', 'd', 'r', 'r', 'd', 'd', 'd', 'r', 'r', 'r', 'd', 'd', 'd']\n",
      "恭喜你，到达了目标点\n"
     ]
    }
   ],
   "source": [
    "maze = Maze(maze_size=10) # 从文件生成迷宫\n",
    "\n",
    "path_2 = my_search(maze)\n",
    "print(\"搜索出的路径：\", path_2)\n",
    "\n",
    "for action in path_2:\n",
    "    maze.move_robot(action)\n",
    "\n",
    "\n",
    "if maze.sense_robot() == maze.destination:\n",
    "    print(\"恭喜你，到达了目标点\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "toc-hr-collapsed": false
   },
   "source": [
    "## 2.4 强化学习算法介绍\n",
    "\n",
    "强化学习作为机器学习算法的一种，其模式也是让智能体在“训练”中学到“经验”，以实现给定的任务。    \n",
    "但不同于监督学习与非监督学习，在强化学习的框架中，我们更侧重通过智能体与环境的**交互**来学习。   \n",
    "通常在监督学习和非监督学习任务中，智能体往往需要通过给定的训练集，辅之以既定的训练目标（如最小化损失函数），通过给定的学习算法来实现这一目标。    \n",
    "然而在强化学习中，智能体则是通过其与环境交互得到的奖励进行学习。     \n",
    "这个环境可以是虚拟的（如虚拟的迷宫），也可以是真实的（自动驾驶汽车在真实道路上收集数据）。\n",
    "\n",
    "\n",
    "在强化学习中有五个核心组成部分，它们分别是：**环境（Environment）**、**智能体（Agent）**、**状态（State）**、**动作（Action）**和**奖励（Reward）**。\n",
    "\n",
    "在某一时间节点 $t$：\n",
    "    \n",
    "- 智能体在从环境中感知其所处的状态 $s_t$\n",
    "- 智能体根据某些准则选择动作 $a_t$\n",
    "- 环境根据智能体选择的动作，向智能体反馈奖励 $r_{t+1}$\n",
    "\n",
    "通过合理的学习算法，智能体将在这样的问题设置下，成功学到一个在状态 $s_t$ 选择动作 $a_t$ 的策略 $\\pi (s_t) = a_t$。\n",
    "\n",
    "<img src=\"https://imgbed.momodel.cn/20200914153419.png\" width=400px/>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "toc-hr-collapsed": false
   },
   "source": [
    "\n",
    "## 2.5 QLearning 算法\n",
    "\n",
    "Q-Learning 是一个值迭代（Value Iteration）算法。    \n",
    "与策略迭代（Policy Iteration）算法不同，值迭代算法会计算每个”状态“或是”状态-动作“的值（Value）或是效用（Utility），然后在执行动作的时候，会设法最大化这个值。    \n",
    "因此，对每个状态值的准确估计，是值迭代算法的核心。    \n",
    "通常会考虑**最大化动作的长期奖励**，即不仅考虑当前动作带来的奖励，还会考虑动作长远的奖励。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.5.1 Q 值的计算与迭代\n",
    "\n",
    "Q-learning 算法将状态（state）和动作（action）构建成一张 Q_table 表来存储 Q 值，Q 表的行代表状态（state），列代表动作（action）：\n",
    "\n",
    "<img src=\"https://imgbed.momodel.cn/20200914161241.png\" width=400px/>\n",
    "\n",
    "在 Q-Learning 算法中，将这个长期奖励记为 Q 值，其中会考虑每个 ”状态-动作“ 的 Q 值，具体而言，它的计算公式为：\n",
    "\n",
    "$$\n",
    "Q(s_{t},a) = R_{t+1} + \\gamma \\times\\max_a Q(a,s_{t+1})\n",
    "$$\n",
    "\n",
    "也就是对于当前的“状态-动作” $(s_{t},a)$，考虑执行动作 $a$ 后环境奖励 $R_{t+1}$，以及执行动作 $a$ 到达 $s_{t+1}$后，执行任意动作能够获得的最大的Q值 $\\max_a Q(a,s_{t+1})$，$\\gamma$ 为折扣因子。\n",
    "\n",
    "计算得到新的 Q 值之后，一般会使用更为保守地更新 Q 表的方法，即引入松弛变量 $alpha$ ，按如下的公式进行更新，使得 Q 表的迭代变化更为平缓。\n",
    "\n",
    "$$\n",
    "Q(s_{t},a) = (1-\\alpha) \\times Q(s_{t},a) + \\alpha \\times(R_{t+1} + \\gamma \\times\\max_a Q(a,s_{t+1}))\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "toc-hr-collapsed": false
   },
   "source": [
    "### 2.5.2 机器人动作的选择\n",
    "\n",
    "在强化学习中，**探索-利用** 问题是非常重要的问题。    \n",
    "具体来说，根据上面的定义，会尽可能地让机器人在每次选择最优的决策，来最大化长期奖励。    \n",
    "但是这样做有如下的弊端：    \n",
    "1. 在初步的学习中，Q 值是不准确的，如果在这个时候都按照 Q 值来选择，那么会造成错误。\n",
    "2. 学习一段时间后，机器人的路线会相对固定，则机器人无法对环境进行有效的探索。\n",
    "\n",
    "因此需要一种办法，来解决如上的问题，增加机器人的探索。   \n",
    "通常会使用 **epsilon-greedy** 算法：\n",
    "1. 在机器人选择动作的时候，以一部分的概率随机选择动作，以一部分的概率按照最优的 Q 值选择动作。\n",
    "2. 同时，这个选择随机动作的概率应当随着训练的过程逐步减小。\n",
    "\n",
    "<img src=\"http://imgbed.momodel.cn/20200602153554.png\" width=400>\n",
    "<img src=\"http://imgbed.momodel.cn/20200601144827.png\" width=400>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.5.3  Q-Learning 算法的学习过程\n",
    "<img src=\"http://imgbed.momodel.cn/20200601170657.png\" width=900>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  2.5.4 Robot 类\n",
    "\n",
    "在本作业中提供了 QRobot 类，其中实现了 Q 表迭代和机器人动作的选择策略，可通过 `from QRobot import QRobot` 导入使用。\n",
    "\n",
    "**QRobot 类的核心成员方法**\n",
    "\n",
    "1. sense_state()：获取当前机器人所处位置\n",
    "\n",
    "> return：机器人所处的位置坐标，如： (0, 0)\n",
    "\n",
    "2. current_state_valid_actions()：获取当前机器人可以合法移动的动作\n",
    "\n",
    "> return：由当前合法动作组成的列表，如： ['u','r']\n",
    "\n",
    "3. train_update()：以**训练状态**，根据 QLearning 算法策略执行动作\n",
    "\n",
    "> return：当前选择的动作，以及执行当前动作获得的回报, 如： 'u', -1\n",
    "\n",
    "4. test_update()：以**测试状态**，根据 QLearning 算法策略执行动作\n",
    "\n",
    "> return：当前选择的动作，以及执行当前动作获得的回报, 如：'u', -1\n",
    "\n",
    "5. reset()\n",
    "\n",
    "> return：重置机器人在迷宫中的位置"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAroAAAHPCAYAAAC8+nn2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAABYlAAAWJQFJUiTwAAARVUlEQVR4nO3dX4jvaUHH8c/j/lMk1ywhlVq1C0OwFs0ULfaAeZOhYRKSoRCW2o1RVBcJuhcVdSMilVagkFJZoKVl+Sf2lOaN4Sok/UN3tcjtItkyc3ddny5m9uxR97jnz8w88/18Xy8Y5tws82GffWbf58d35jfmnAEAgDYPWT0AAACOg9AFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCg0tWrB1yOMcbrkrx29Q6O3dkkN60ewbFyxvvgnPfBOe/DzXPO160ecbE294ruYeSeWTwDOBo3rh4AHJkbVw/gRJw5bLFNGHPO1RsuyRhjW4O5El4dgA7uMpSZc47VGy7GJh9duM9W/iUDX+/8v7S6y7Bt7vM+bPHFxs09ugAAABdD6AIAUEnoAgBQSegCAFBp0z+MVmGMkeRRSZ5w3sf1Sa5Jcu3h52uS3JvkniR3H37+UpJ/S/Lpw4/PZs4vn/R8AIDTSuielDEemeRZSb4z9wftEw8/f9MRfIV7M8Zncn/4fjrJp5LcmuQfs7XfIwcAcIU2/Xt0T/WvMBnj2iTPTPLcw4+nZ92jIv+e5ANJ3p/kA5nzjkU74JzN3GXgQbnP+7DFcxa6R+XgEYQn5/6wvSnJw5duurBP5CB635/kbzPnFxfvYYdO7V0GLpn7vA9bPGehe6XGeHSSn07yiiTfvnjN5bg7ybuSvCHJRzziwEk5dXcZuGzu8z5s8ZyF7uUP+e4kr07ykiTXLdtxtD6ag+B9R+a8e/UYup2auwxcMfd5H7Z4zkL30r74VUmen4PAbX7f9s8leVOSN3mel+OyxW+YwANzn/dhi+csdC/ui16X5FU5CNzHn8jXPB3uTvKHSX4lc/7z6jF02eI3TOCBuc/7sMVzFroP/gWfk+Q3kzzp2L/W6XV3kt9I8mt+cI2jssVvmMADc5/3YYvn7J3RLmSMx2SMP8jBr+Xac+QmB29c8Zok/5Axfnj1GACAiyF0v9YYI2P8RJJPJnnx6jmnzOOTvDtjvC1jPGr1GACAb0Tonm+MhyV5a5LfT/LIpVtOt5ck+VjGeNrqIQAAFyJ07zPGDUk+lOSlq6dsxHck+XDGeNnqIQAAD0ToJskYz0ry90meunrKxlyX5K0Z4/UZw39LAMCpIk7GuCnJ+5J8y+opG/azSd4sdgGA02TfYTLGDyZ5b5KHr55S4OVJ3nL4phoAAMvtN3THeHqSdyd52OopRV6a5I2rRwAAJHsN3TEek+RdSR66eEmjV2WMV64eAQCwv9Ad46FJ3pnksaunFHvj4bPPAADL7C90k9cmecbqEeWuTvL2jPGI1UMAgP3aV+iO8T1JfmH1jJ14XJJfXT0CANivMedcveGSjDHODZ5zjkv4B69K8pEkTz+GWTywmeTZmfMjq4dw+lz2XQZOHfd5H7Z4znt6RfeFEbknbcSrugDAIvsI3TFGkl9aPWOnzmQMz0QDACduH6GbPCfJ01aP2DF/yQAATtxeQvcnVw/YuRdkjG9dPQIA2Jf+0B3jmiQ/tHrGzj0kyfNWjwAA9qU/dJObkly/egR5weoBAMC+7CV0We/M4Q8FAgCciD2E7pNWDyBJ8s1JPKcLAJwYoctJchYAwInZQ+g+YfUAznEWAMCJ2UPoPnT1AM65bvUAAGA/9hC6fgDq9NjDf28AwCmxh/D4v9UDOOdLqwcAAPuxh9C9ffUAzrlt9QAAYD/2ELr/tHoA5zgLAODECF1Oyp1J/nP1CABgP/YQun+zegBJkrOZc64eAQDsxx5C95Yk/7N6BPnT1QMAgH3pD90570ry3tUzdm4mec/qEQDAvvSH7oG3rh6wc3+eOT2fCwCcqL2E7l8m+cTqETv266sHAAD7s4/QPfghKLG1xocz54dWjwAA9mcfoXvgHUk+vnrEDv3y6gEAwD7tJ3Tn/HKSlyf5yuopO/J7mfPs6hEAwD7tJ3STZM6PJnnD6hk7cUeSX1w9AgDYr32F7oHXJPnY6hHlvpLkpZnz86uHAAD7tb/QnfOLSX4k3o72OP185nzf6hEAwL7tL3STZM7PJHlhkntWTyn0lng8BAA4BfYZukky54dzELt3r55S5I+TvOLw17kBACy139BNkjnfk+T5Sb60ekqBtyf58czpVXIA4FTYd+gmyZx/leR5Sf579ZQN+90kLzv8FW4AAKeC0E2SOf86yfcm+eTqKRtzT5KfycHjCveuHgMAcD6he585/yXJM5L8yeopG/G5JGcy5297JhcAOI2E7vnm/EKSH0vyyiRfWLzmNHtnkhsz59+tHgIAcCFC92vNOTPnm5M8Jcm7V885Zf4jyYuT/GjmvGP1GACAb0ToXsict2XO5yd5QZLbV89Z7N4kr0/yXZnzjzyqAABsgdB9MHP+WZIn5+Ctgz+3eM1JuzcHzyw/NXP+XOb0mykAgM0YW3txboxxbvCcc5zwF782yYuSvDrJ953o1z5Zn0/yO0l+6/Bd5ODILb3LwJFyn/dhi+csdC9/yDNzELwvSnL1sh1H65M5ePvet2XOL64eQ7dTc5eBK+Y+78MWz1noXqkxHpeD3yX7U0kevXjN5fhKkr/IQeB+0PO3nJRTd5eBy+Y+78MWz1noHpUxrkry1CTPPfx4dpJrlm66sH9N8v7Dj1sy5+cX72GHTu1dBi6Z+7wPWzxnoXtcxnh4kh/I/eH7lIVr/ivJB3Nf3M5528ItkGRDdxl4UO7zPmzxnIXuSRnj25J8f5LvTPKEw48nJrkhR/fK7x1JPn3ex6eSfDzJx7xFL6fNZu8y8HXc533Y4jkL3dUOHnl4bL46fh+Rg/i99rzPX05yT5K7Dz/fleSzuT9qb8uc/3vS8+Fy1d1l2DH3eR+2eM5CF1jCXYYe7vM+bPGcvWEEAACVhC4AAJWELgAAlYQuAACVhC4AAJWuXj0ALmSMcUuSm1bvAK6MuwysssXQPbt6AHBk7k3yhdUjgCNxZ5JbV4/gWG2uwbYYul4VgB5XJbl+9QjgSNw65zyzegTHanMN5g0jgCXcZYBt2eL3bT+MBgBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQKWrVw+ACxlj3JLkptU7gCvjLu/GnUmuXz0CzrfF0D27egBwZO5McuvqEQBctE19395i6HpVAHrcOuc8s3oEABft+myoxcacc/WGSzLGODd4zjlWbgEA2IstNpgfRgMAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoNLVqwdchrOrB3D8xhi33PfnOeeZdUs4Ls54H5zzPjjn3dhcg4055+oNl2SMcW7wnHOs3MLxcc79nPE+OOd9cM77sMVz9ugCAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVrl49AB7MGGOu3gDAxRk3+57N6bHF0D27egAn4mySG5Ncv3gHx+veJF9YPYJjdefqAZyA61YP4ETccPj59u202BZD96bVAzh+c84zY4xb4rzbXRV/mWnnfPfgrtUDOBG3n/vTZv7fvMXQZSfmnGdWb+D4eCQFgOPmh9EAAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoJHQBAKgkdAEAqCR0AQCoNOacqzdckjHGtgZzJc4muWn1CADgq805x+oNF2OLr+jenIMAArbvztUDOBHOeQ+uWz2AE3FDkoMW24TNhe6c83VJblk8Azgat64ewIm4dfUATsBdXoTahdtz82GLbcLmHl0AAICLsblXdAEA4GIIXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACoJXQAAKgldAAAqCV0AACr9PzgMHitz8lVXAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 231,
       "width": 349
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Maze of size (5, 5)\n",
      "the choosed action:  u\n",
      "the returned reward:  u\n",
      "table:  {(0, 0): {'u': -5.0, 'r': 0.0, 'd': 0.0, 'l': 0.0}}\n"
     ]
    }
   ],
   "source": [
    "from QRobot import QRobot\n",
    "from Maze import Maze\n",
    "\n",
    "maze = Maze(maze_size=5) # 随机生成迷宫\n",
    "print(maze)\n",
    "robot = QRobot(maze) # 记得将 maze 变量修改为你创建迷宫的变量名\n",
    "\n",
    "action, reward = robot.train_update() # QLearning 算法一次Q值迭代和动作选择\n",
    "\n",
    "print(\"the choosed action: \", action)\n",
    "print(\"the returned reward: \", action)\n",
    "print(\"table: \", robot.q_table)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "toc-hr-collapsed": true
   },
   "source": [
    "### 2.5.5 Runner 类\n",
    "\n",
    "QRobot 类实现了 QLearning 算法的 Q 值迭代和动作选择策略。在机器人自动走迷宫的训练过程中，需要不断的使用 QLearning 算法来迭代更新 Q 值表，以达到一个“最优”的状态，因此封装好了一个类 Runner 用于机器人的训练和可视化。可通过 `from Runner import Runner` 导入使用。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Runner 类的核心成员方法：**\n",
    "\n",
    "1. run_training(training_epoch, training_per_epoch=150): 训练机器人，不断更新 Q 表，并讲训练结果保存在成员变量 train_robot_record 中\n",
    "\n",
    "> training_epoch, training_per_epoch: 总共的训练次数、每次训练机器人最多移动的步数\n",
    "\n",
    "2. run_testing()：测试机器人能否走出迷宫\n",
    "\n",
    "3. generate_gif(filename)：将训练结果输出到指定的 gif 图片中\n",
    "\n",
    "> filename：合法的文件路径,文件名需以 `.gif` 为后缀\n",
    "\n",
    "4. plot_results()：以图表展示训练过程中的指标：Success Times、Accumulated Rewards、Runing Times per Epoch\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "**设定训练参数、训练、查看结果**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "正在将训练过程转换为gif图, 请耐心等候...: 100%|██████████| 342/342 [00:06<00:00, 56.27it/s]"
     ]
    }
   ],
   "source": [
    "from QRobot import QRobot\n",
    "from Maze import Maze\n",
    "from Runner import Runner\n",
    "\n",
    "\"\"\"  Qlearning 算法相关参数： \"\"\"\n",
    "\n",
    "epoch = 10  # 训练轮数\n",
    "epsilon0 = 0.5  # 初始探索概率\n",
    "alpha = 0.5  # 公式中的 ⍺\n",
    "gamma = 0.9  # 公式中的 γ\n",
    "maze_size = 5  # 迷宫size\n",
    "\n",
    "\"\"\" 使用 QLearning 算法训练过程 \"\"\"\n",
    "\n",
    "g = Maze(maze_size=maze_size)\n",
    "r = QRobot(g, alpha=alpha, epsilon0=epsilon0, gamma=gamma)\n",
    "\n",
    "runner = Runner(r)\n",
    "runner.run_training(epoch, training_per_epoch=int(maze_size * maze_size * 1.5))\n",
    "\n",
    "# 生成训练过程的gif图, 建议下载到本地查看；也可以注释该行代码，加快运行速度。\n",
    "runner.generate_gif(filename=\"results/size5.gif\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABYYAAAIPCAYAAADHDgrqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAABYlAAAWJQFJUiTwAADxwklEQVR4nOzdd5hbZ5U/8O+Z3os9xfaMa2LHiRM7dmI7hBRD6B2SbEKAJezCwu6PunRYwCzLUnepS2dhgfRQA6GkOYUQ24kndppL4jYztqd4PL1L5/fHe6/0SpZmNDOS7pX0/TzPPBqVK72jsedKR+d+j6gqiIiIiIiIiIiIiCh35Hm9ACIiIiIiIiIiIiJKLxaGiYiIiIiIiIiIiHIMC8NEREREREREREREOYaFYSIiIiIiIiIiIqIcw8IwERERERERERERUY5hYZiIiIiIiIiIiIgox7AwTERERERERERERJRjWBgmIiIiIiIiIiIiyjEsDBMRERERERERERHlGBaGiYiIiIiIiIiIiHIMC8NEREREREREREREOYaFYSIiIiIiIiIiIqIcw8IwERERERERERERUY5hYZiIPCci14uIisg2r9dCRESZTUS2OfuU671eSzJxX+k9/g6IyOX8LVARWeb1WlKJf/coU4jIYeff6hav15JpWBimjCMiBc4O6k8iclxExkXklIg8IyJ/EJGPicgmr9eZS0Tkp9aLo5l+LfN6/UREqSQir7P+5t3l9Xpo5kTkfBHZmonFZhFZFmf/OyEiHSJyl4i8XUQKvF4rEeWeKd5HDIjIUyLyHRE52+t1Ziq+T6MZ/s5rvF4vpR9fAFJGEZF6AHcCuNC6eBSAADgLwGoArwDQB6Am3evLYX0AOmJcXgqgyvk+1vUAEHC23wfgaPKXRkTkubda379QRJpUtd2z1dBsnA/gMwDuB/BTT1cyN6cAjDvflwFoAPAi5+utIvJSVR32anFElNMmAPQ43wuAOgDnOF//KCJvVtXbvFpcDPuc0wlPVzE9vk8jVz+AkWluE0zHQshf2DFMmeYXMEXhAQAfAbBQVUtVtQZANYAXA/gOgF6vFpiLVPV9qrog+gvA+6zbnHa989Wqqr9W1dWq+vce/hhEREknInUAXglgCMCNMK+93uLpoiiXvcHa/1YBWATgf5zrLgGw1bOVEVGue9j6+9QIoATAywEcBlAE4CdOk5AvOO9dVvv9g16+TyNLzH8LUV/9Xi+S0o+FYcoYIrIawEucs/+gql9R1RPu9ao6oKp3q+r/g+kcJiIi8tobARQC+B2A7zuXvTX+zYnSR1WPq+q7AbgRJ/zQgoh8QVUnVPVPAN7kXFQO4EoPl0RElJVYGKZMcp71/e+nuqGqjkZfZuUrbY233XQDa0SkUET+SUTuEZEuERkTkSMi8hfn8vI4271MRG4XkTZnmxMi8oiI/JuILI6zzbki8r8ickhERkWkV0T+KiLvEpHCONs0iMhXRORJERlytmsVkYdF5N9FZGmMbV4rInc6OYMTItIjIvtE5CYRuSbec5VMMsVQA7FC5EVkoYh8z/mZRsTkSn9ARPKs218tIg86z1e/mNzpc6d5/HoR+YKIPCEig85z96SIfF5E5sXZpkhE3uc8t70SzmrcLSL/IyLPm/MTQ0TZwC0C3wDgQZhDMVdLAln4InK28zdvv4gMO39rnhCRb4rIBXG2mS8inxWRx5zbDzvb3ywir4u67Vbn7+tPp1hDzH2n8zdZReSwc/6lInK3sw/pFZNb+zzr9tXO39T9zt/vVhH5koiUxnjMUCbuFOuKePxEicgGEfmiiDwkIkedffJJMfv/t4tIfoxtFMBPnLOXy+l5fFtibHOJ85y3WY9xt4i8UURkivUtEpEfiEi7sw8/KCL/LanP/PuLc7pgiv1ewq9LRGSx89xMikhVjPt6QsIZorGe8+PRz62I5IvIy0Xk+86/7w4xcyaOicivReSF8X44sV7fiUiN829vr/v/Kuq2s/odiEiliHzKWduAtbZHxbw2m/K1CBHF9TcAg87359hXJHk/9nwR+b2IdDv7qd0i8u54f7OtfcCyqdYkIm8Vke3O34V+EblPRF481Q8sIueIyC0i0umsZa+YfXtJIj9zMgnfp0XvQ2pF5GvOvmFUzH7+ByKycJr7qBCRT4jIThHpc7Y9IOY1Xbx6QML7rlSQqNdj1v+RLmcNjzv/R6asK4rIG8TMh3LrJ20icoOIbEhgDTOuozjbzROz7z7kbNcuIj+c7veUs1SVX/zKiC8AVwNQ5+uMWWz/U2fbrVPcZptzm+tjXNcEoMVaQwDASQBj1mVborYpAvBz63qFibkYtM6fth4A73bu373NAIBJ6/x9AMqitlkK4Jh1m0mYnK6gddm7orb5fNTa3Nwh9/yJOf7OrnfvK8HbbYtx3WHnurcBOO583xf1fHzLue0XrZ+937r+FICVcR77Euf36N52LOo5OArgrKhtCqx/K+o8x6ei1nSz1/9n+MUvfnn7BWCN8/egG0Chc5n7d+p/ptn2PVF/UwadvzPu+W0xtrnUeSz779lJ+36ibr/VufynU6zjp7H2VQC2OJcfBvAvzt9BN4vQffwRAM8HUA/gCevnsPebv4/xmMum23fYjx/jOvfv8/UxrrOfn6Go51QB/AFAQdQ2J6yfa9w5b39dHHX7L0XdZx8i98U3AciLsbazAXRG/c6Hne8PAPjXeL/7af4tLbPuc0uc23zYuk19jOtn87rkoHPdy6Munx/1fGyMun6Vc/kogBLr8nNjPK+DUZd9PM7P5/6b+DCA56z77wfQO9ffAUyc2VPWdgGY12D2c/ZFr/8m8YtffvxCeD+zLc71Yv1f/5+o67YiOfux652/aUGY92r235Wvx7lf9/pl8dYE4EcIvzex948BAFfGud8XIfK9SB/C+82/AfjCdD9zgs/79e5jJHi7034/yJH3adb9fRDAs873w4jcB3UCODvO9mdbz5XC5FLb2/YAeP4UjzvlviuB9buPc/0Mf+5l1rZXOut2f2cT1nW/RtRrJ2f7PAD/Z91uEpGvuwIA/jnOY8+4jmI9x2+2vh9ynjN3m0MAaufyfycbv9gxTJnkMev7/5E0ZkyJSDGAO2CGz3TDdIBVqep8mOEtFwD4Ok4Pc/8azB+mAIDPAligqjWqWgFgBcwf+WNRj/U6AN+C+SP2EZg3aJXO47wM5o3JFue+bZ8BsBBmZ3UZgCJVnQczWOA8AP8B8wbWfZxlAD7mnP2C8zhVqloKM4zmKpg3yH7xNZg/5OtUtRpmWMKnnOv+n4h8AuYN2/sBVKvJTjwPZlhCDUwRPIKYDuo7AMwD8F0AK2Ger3Jn278AWAzgVxLZ0XQdgMthXhC8BebNcC2AYpgC/bsB7E7Sz01EmeutzumtquoOp7nBOb1WRIpibSQiVwP4JoB8ALcDOEdVK5y/M/Nh9iuPRW1zBszRNPMBPA7ghTB/m+YDqISJYvpVkn4uWz3M3+cvAJjv/H1eDvPmtcS57nswcRqXOmupBPB2mDcIrxSRV6RgXfH8BSbeY6GqljvPaQXM3/ITMANsP2BvoJFZjHYGpvv1sHtbEXkfzL67A8A/AahxnpNyANc6j3EtgI/ajyGm4/Z2mOfzIIDLndcKFQBeA1N4/HTynobTuFFdg6raFbW212F2r0secE4vj7r8MphCz0Cc693zOzTyCLBxAP8L4KUw+/lq5zlqhHk9EADweRHZPMXP+WmYf4svh/n/UQVnoPEcfwfvg+lk7ALwKgDFzmuwEphC98dg3tQT0cxdDPM3FDD/N5OtHibq6bsw+4YaALUwf/cA4L0ismYW9/tamBiMf4Z531gN8/7vAZiC2bdEpMDeQMxcgpth/nbsAHCes12Fc1/nAnjXLNaSarnyPu1TMK9hXg2gwtlHbHF+9noAt8npR9BUA7jTeezbAKyD+dCzAsAZMPMnagH8UuIfmRJ335VGPwZwN4AVzvNZA/OaIAjgdc730T4C4O9hCrKfginI1gJohnku8gB8W0Qui7HtjOsolm/BFKAvVtVymP8/r4UpLC8D8PHEf+wc4XVlml/8mskXIj9xGoP54/QfMP/RT+twidr2p4jxyVLUbbYhxqdpMN1Q7id0axNc6xqEO2L+KcFt8hH+dOulcW5zBsybswmYFy/u5U87212T4GP9nXP7Z1L4+7re/X0leLttMa5zn48emDfY0dffY/2b+HSM6y+1fndFUdf9wrnuC3HWVQTzwkEBXGVd/h3nsu+m698+v/jFr8z6cv6eu0dxXBJ13R7n8tO6hWBe+Lc51984g8e71dlmH4DKBLfZirl3WimAn8TYbom1DxwHcGaM2/zYuf5/oy5fNt2+A7PsGJ7m+XD3F4diXBd3P2Xdpgam2DkC8+Y41m2e5zwvPfY+CebNq/va5qwp1jblGuI85jJr2y1R1y2EeQPlXv/dqOvn8rrkbc52f4u6/dedy92jlu6Iut7dN39uhj/np6b49+j+mxgHcG6c7Wf9O4B5068APjqTNfOLX/yK2M9si7q8EOaDoEPW/9/mqNtsRXL2Yz+Ms627v471HsPddlmcNSmAN8XYbhHCHcCXRV33WefyDsR+3/N31n3H/ZkTfN6vd+8rwdtti3Gdu3/I6vdp1j4kiKjXdM71Z1m/0zdHXfcfmOY1HYA/Orf5UJzHjbvvSnD97u+gD6cf9WR//SRqu2XWtk/CfOgZfd9brfsusy6vQLhL/rTfIczriwed6x+Ium7GdZSof48nYJoVoq//oHP9wWT928iWL3YMU6Z5B4D/hvnjWATgCgCfBPAbAJ0iskNE3iQSP79vltwprD9R1T0JbvMWmI6Yvar6gwS32QLzaeKTqvrnWDdQ1ecAPAJzmMwW6yp3gmiiuTnu7atFpCzBbbz0PVXtjXH53c7pOMy/jWh/hXmxUQzgTPdC52e+GmanE2s7qOo4TPcQANhZYDN9roko97wY5m/EEZi/Qza3a/itMba7Aia6KADTDTEtEakA8Hrn7KdVdWCq26fAF6IvUNWjMJ2kAHCbqj4bY7t7nFNfZK+q6oNwuklEZNEs7uJKmDdCd6tqzG4kVf0bnMMYYY42cl3lnP5KVffFWdsD0ZfPwq+cfL4TItIP8+HFu53rdgH4RNTtt2D2r0vc9V4okTMYLndOvw3zfF8SlU/oXn9/gj+T6w7n9PlT3OaPqvpknOvm8jvg6wKiubvY+vvUAfP6/U8wxakggHeqaluKHvu0/Zjjt87pbPZTR2G6QSOo6jGYbuBY9/sG5/QHsd73qOqtSE3X9Fzlyvu0B1X1oRhr2Wet5aqoq93Xev81xf26/07iZU9Pte+aiSqYo2zifdVOse1/qepYjMv/G+Z3WIXw0UeA+VmqYH73X47eSFUDAD7nnL1URBZYV8+mjmL7gaqejHH5b5zT5RJnNlSuYmGYMoqqjqvqB2EOG3kXTE7fAZhPfgBgI8yni7dMF4KeKOdwEPfN250z2PSiWWxzsXO60nphdNqXdTs7cN19nC85ofovkBhDfSzbYT7dXQjgb2KG5y2fwVrT7Yk4l3c6p4dVdTD6SlUNwsR/AJE7uwtgPlwQAE9M8Vx/yLm9/Vz/0Tl9rYj8zgnUnz+bH4qIstb1zulN6rQpWG6C2W+9PEYskrvv2K2q7Qk+1oUwRTmFeROdTqMIF4CjuX+f472Z6XBOp3ojknRiht/8RszwuRGxBsnBdP0CpqNrptx98wun2Ye7+xN7v+IOYJmqGDrTQmkstQi/Aay0Lv8xgOep6qmo28/6dYlTMG6H+bd5MQA4h8muhXmzdxymW6gG5tBaiMgKmENMJwCEIjpcIlIqZpjRNjFDmSas312Lc7Opfnd/m+K6ufwO3Ndg7xWRn4sZklcZ57ZEFFshwn+fGhCuVfQA2KyqP0nR4/aoarxiq7sfns1+6tEY+/+49+tEF7rD9U4rPlqmus4rufI+bdsU17n7h9BANWc4WrNz9s4pfo5vOLeJN0xtqn3XTLxNVWWKr9dNse22WBeqaj/C+197mJz7/e4Yry1cD8A0QkRvO5s6im1nnMvt19U1s7zvrMTCMGUkVe1U1e+r6nWqugqmuPkOAK3OTa6GGdyTDPNg3tQA5pPfRDXOYhv3k81iTP1pXolzO7vT90sAfgezE/0XAPcC6BczjfXD0ZlFzh/ot8Dk76yFydY6KGYS+P+JyOXwl+NxLg9Mc719GzvzyX2uBVM/1+409dBzrar3w2Q9TcJkTP0SQLeY6btfFZGVifxARJSdnDy51zpnY3ULHYUpiBXAZOHZZrPvcLfpU9W+GWyXDB1TvPGd7u9zrL/NKSMiBSLyK5jYjdfCvAETmDelHc5X0Ln5bDpJ3P1KGaberxRat3O5HxDEy8sDIt/QzNYL3DeAABbAfIBxEsA/IHYH+1xelwDhN8rua4pLYd5/bItzvXv6mKoO23ckZpL44zDdSZfDPGdjMLm+HQgXF6b63XVNcd2sfweq+jMAP4D59/RmmDezvSLSIiL/LpyCTpSI+62/TyUws11uh3kv9mMRSdWHiFMdZePmnM9mPzXT+61FuD4z1fuaqf5GeSVX3qdNtR92r7M/8Lf/9jdM8XO4/7bjHcU71b4rXWb6s9dHXXcaNXME3H23ve1sXgvbYv7f08i5BWl57ZkpWBimrKCqHar6I5hPmtwOpH/wcEmz5f6f/O00n+a5X1vdDVV1TFVfC5Nf+GWYwzrVOr9fRNbZD6aqd8IMCfonmDfKx2DeKP49gG0iMptDNzKF+1z3Jfhcb7E3VtXPwQyU+TiAP8MctrQaJrvoaRH5exBRrroG4ULZHrsj1epudAdtxCrGUWq8AyZyYxjAewEsVtUSVa1XZ5Acwm+4ZxNJ5e5XvpHgfuWnc/6J5sB57fR/MENjADMA5vyom836dYkjegBddExEvMJwrM7cr8Psdw/CxHbMUzOUscH53V0UY5togelvMjuq+k6Yw8L/HabwPQZT2PoUgAMiEu8QYSKK4ryv2Q2TqftnhJtYKDdky/s0u95Wm8DPsSzO/aRs35UGJdPfhLzGwjBlFVXtRjgLalXU1ZPO6VR/nKpjXNZjbbt0BstxC9Sz2WbJDLaJoKqPqOpHVfV5MJ8+vhHm07Z6AD+Kcfs+Vf2hql6jqk0wYe8/dK5+h4i8crZr8Tn3ua5yuvtmTFUPqeoXVfVlMN0ML4B5E1wA4Dsi0pCcpRJRhplJsXe9iJxnnZ/LvqN6hn/PZrtfTDV3XRCReGubzbqudk4/p6rf0qisSjFTzetmcb+uuezD3W6gqWIQZhNvMS01eYm/gDni6GtRV8/1dYlb4N3kxFu5hd9tzmkLTGfPpSIiiFMYFpEihLvw36Sqv4pxaGoj5mbOvwNVfUpVP6OqL4A5TPXVMIdYlwP4P4maVk9EU3OOSHkvTGHs6hhHNPp1PzYbpxA+amWqowxy4QgEv75PS2T/YHf3dljfz/r9vU/M9Gd3v4/7czuv8dyYj1jP20xeC9McsDBM2WjIOR2PurzXOW1GDE4A+dnRl6vqBIDHnLOvmME6HnFOXz6Dbdz8oLUi0jSD7WJS1SFVvRmmIxgALpBpgtZV9WlV/SeE1++3SIlkeRTmxaQAeNlc70xVA6q6DcCrYLIRy2FyP4kohziHKLp5q+fDfEAX78sdlmUXkt2/vTPZD9h/z2ayz+l1TuPtFwWRA9LSpdf6PubaYGYKzJR7Xy1xrn8+4hcX3DfrU3USu/vwLdNk/Meyyzm9bIrbpHJ//HmYo4y2iMiLrMvn9LpEVffCZEwWwQylWQ9gv6qecK4PwAwfmg/zGmsZTAEoemBjHUycBRD/9/eiOJcnKqm/AzVzMX6P8AcSCwEwaopohlR1P4BbnLOfj7q61zn1235sxtQM9nraOXvJFDe9NA3L8Zpf36dNtQ9wr3P3JVDVQwgXOWfy+syPYv7sTp6+mw+8y7rK/X7lFK8fLkM4stPedjZ1FJoDFoYpY4jIchE5Y5rblCF8SOTjUVe7ofgvidOB9AGE33RE+5lzer2IrJ1+tQCAn8O8yVotIu9McJt7YHKS8wF8ZaobRudsOd008Yy4N4N5czbd7e1t4j0nGU1VB2AypwDg36caEuPkUlZY56d67sYRPtwnK587IpqSe3jiblXdraq98b4A3Obc9k1Otypg9gPtSGA/4FIz0OXXztnPzmDolbtf3BgnA/VNiD8IJWWcn+ewc/a10dc7Q2TePou7dvOXz4u+QkQKAPzHFNu6U85rprjNbTAfTtfC5BvGFSMr0/238IZY+YcicjGmLljOiZqJ6r9zzv6bddWsX5dY3DiJTzr3sy3qerc7+DPOaYuaYTa2AYQHDcf6/S3E3GdLzPp3kOBrMICvC4hm66vO6fNFZIt1uS/3Y3Pg7svfEatTVkSuBLAivUtKPx+/T7vc2RdEr2ElgKucs7dFXf1T5/RDU33AKkbNLNaULh+M89y+H+ZD9X4Af7Eu/4tzWSGAD0dv5Lzu/ZRz9kH3A2PHbOooNAcsDFMmWQNgn4j8SkT+zt75i0i5iLwaZpjPcufib0RtfwfMi/N6AD9zDx8RkWoR+SSArQi/aYz2Y5hCczGAe0TkLU4RGiKSLyIXisgPRWSzu4GqPoVwFtb/iMhW+5AVp9C9VUTeZW0zAeDdMH8I3yhmavr51jaFzmN9GcChqDU+KSL/KSIb3T/azg5mE4BvObfZaR16+c8i8mcRuS7quawRkU8A2OJc9Oc4z0k2+BhMVMgqAA+LyMvcwzyd526liPwrgL2I/FT5ZyLyExF5qf1CRUSWAfg/mJ3jCMy/RyLKEU5n0lucs79KYJM7YDpXFgB4KRDaD3zQuf6NInKriKy2HmOeiLxDRL4ZdV+fgCmerQLwgIi8QETynG1KReSVIhI93fmvMJm6RQBuEpHlzu3LnBfiP4Q5tNULtzqn/yYir3EKtxCRiwDcDedDzhm6yzn9lIi81i3GO8/vHQA2IXzUUbSnnNNz7H29TVVPwuQZAsDHnNcFoVgr5/dwqYh8F8DDUZvfAtMpVgwzufwSZ5s8MZFOv0K4OJ0qbuH3cvfx5/i6xOUWht0u7+j84Punud4tErgdRP/rrsF5fq5wtplNLrRtLr+Du0XkmyJymVjd4iKyBuGiwHGEi1hENAOq2gLztx+I/PDKz/ux2fgWzHobAfzR+RviFj+vBfATRB5Vk838+D6tH8CvROQVzms+iMilAP4Is+94CuHXL64vwmTj1zk/x99F7SeWiMg/wXTMvm4Wa0qXJQB+7TyP7v+xD8LUUADgS2oNjVXVIQD/6Zx9r4h8UpwCvpgC+U0wnfFBRP6fnlUdheZIVfnFr4z4gnnTrFFfwzA7R/uySQCfiHMf74267SmYTw0Vprtnm/P99TG2XQzzgt5+nG6Y4SLuZVuitimGeaMR/ZiD1vmtMR7rbVH3OwwzNXzSvq+obeznYdK5/bh1WReAtdbt3x+1rkFnbfZl35/j7+z6WGud4nbbYlx3ONZzm8i2idwHzBvRdutnHo/xe1UAl1vb/Ma6POg8b0NRz/9bvP4/wy9+8Su9XzD5de7fgTUJbvMn5/a3RF3+r9b+SWGKvvbf6G1xHt++jTvtObTviLHN66Mepw+mWK0wH4r+NNa+CubDQwVweIqfbRvi7FOnuw+Yrtvnon4Wd995BMCbp9g25uPCZAw+G/X3vs/6u339NPuL+61tTzq3PQzgoqjb/Zuzb7D3rz1Rz/OhGPd/Dkzsgv07H3a+P+D8m5hyfxfneV5m3edpP1fUbf/q3O7PUZfP+HWJte1a+zYAFkVdX4jIfeir49zPZuv5cJ9X9/xJmO7yeP/Op/y3ONffAUzzgLtNwPl9j1iXDQG4Itl/c/jFr2z4Qng/s22a273Y+j91kXV5Kvdj18dbm/V4y6Iu3+pc/tMEfuatMa57Kcw+z73/Xuv8QwC+AL5PS+v7NIT3IR9E+HXEMMJHsyjMvuOcONufCfPBo72GbkTu0xTAW+M87vVz/F3b/zdOTPN1sbXdMmvbKxH+f3XK+l6d57wgxuPmwxTj7Z+7B+HXSAEA/xJnzTOuo0z373Gq/7e5/sWOYcoYqvpnAGcB+BDMH59nnasqYHaYu2AmVq9T1f88/R4AVf0mzLT4R2D+EOfBvAl6var++zSP3wrzaeR7YXbKA85jH4fpqn07gB1R24yp6jUwb1bugMkYKne2fQTmsMofIoqq/sT5Wb8O88ljAEAVzBufbTCHW54VtdlrYV4ouJ+cV8DsPPfAfFK5RlX3WLe/EWZC+y0AnoH54+7+PL8D8Bo1E7azmqruhJlS+1GYDq5BmEOFh2Hyrb4J82LD7mD6GICPwBR0DsJ0KeTDFDF+AmCDqv48TT8CEfnHW53T/Wq6HRLhHir5GrEOIVTV/4bJY/0JzAvdQpgXsntgjoj5QPQdqep9MPuGLwF4EuYFeAnM36abALwmxja/hsl+vQ9m35QPU+T6R1X9xwR/hqRTc3TLxQB+ALNPy4PZB34LJsuuLf7Wce+zB8BFAL5rbT8C85riclX96TR38QYA34HpjK2AGYqyFFG5xKr6HwDWOWs/4Ky9HOHXCx9BjIxIVX0aJpf6R85tC2HepH0N5s1xT8I/7Ox92Tl9iYiEcpxn+brE9QTCa39WVY/ZV6rpSnY7qIMwr7FOo6rbATwP5vd1Cub56YTpKjofwO5EfsCpzOF38HaY5+A+mIG/bjfYXgDfBnCuqt4z1/UR5TJVvQvhjPFPWZf7cj82W8573gsB3A7zN7YYZr/zGQBXIPz3pdeL9aWTD9+nnYQ5uujrMO/ri2Beo/wQwPnOPiTWz/EszGu6f4H5d3oKZijiJMzruh8AeCXMINhUqoLpRp/qK+YRWar6S5gGhD/AvAaYhNnvvgfAG1R1MsY2AVV9K0zMxl9g/s269YabAGxS1e/EebxZ1VFodsSpmhMRERERERERkU+JyIMwh+C/LYEPNCkJRGQbzPC1nHrOndiIQwCgqnONayIfY8cwEREREREREZGPicjzEM5l5VEIRJQUBV4vgIiIiIiIiIgo1zmDyOpg4v4Oq2rAGdr1BphYGwC41Yk5JCKaMxaGiYiIiIiIiIi8twQmP/XzAAIi0geTq+se7f04TK4rEVFSsDBMREREREREROS9m2EGzF0OoBnAPAD9AJ6GGUj3PVUd8W55RJRtOHyOiIiIiIiIiIiIKMdw+BwRERERERERERFRjmFhmIiIiIiIiIiIiCjHsDBMRERERERERERElGNYGCYiIiIiIiIiIiLKMSwMExEREREREREREeWYAq8XkE4icghAFYDDHi+FiIj8aRmAflVd7vVCMg33sURENI1l4D52VriPJSKiaSzDLPexOVUYBlBVWlo67+yzz57n9UKIiMh/nnnmGYyMjHi9jEzFfSwREcXFfeyccB9LRERxzWUfm2uF4cNnn332vMcee8zrdRARkQ9dcMEF2LVr12Gv15GhuI8lIqK4uI+dE+5jiYgorrnsY5kxTERERERERERERJRjWBgmIiIiIiIiIiIiyjEsDBMRERERERERERHlGBaGiYiIiIiIiIiIiHIMC8NEREREREREREREOYaFYSIiIiIiIiIiIqIcw8IwERERERERERERUY5hYZiIiCgDichhEdE4XyfibHOxiNwpIj0iMiIie0Tk/SKSn+71ExERERERkbcKvF4AERERzVofgK/HuHww+gIReS2AXwIYBXALgB4ArwbwNQDPB3B1ylZJREREREREvsPCMBERUebqVdWt091IRKoA/BBAAMAWVX3UufxTAO4FcJWIXKuqN6dysUREREREROQfjJIgIiLKflcBqAdws1sUBgBVHQXwb87Zf/ZiYUREREREROQNdgwTERFlrmIReTOAJQCGAOwB8ICqBqJu90Ln9E8x7uMBAMMALhaRYlUdS9lqiYiIiIiIyDdSVhh23qj+3Dn7DlX90Qy2PQfAVgBbAFQBOALgZgBfVNWR5K6UiIgoYy1AeF/rOiQib1PV+63LznJO90ffgapOisghAGsArADwzFQPKCKPxblqdWJLJiIiIiIiIj9ISZSEiCwG8G3EGH6TwLabAewE8DoAdwP4BoB+AJ8GcJeIFCdvpURERBnrJwCugCkOlwM4D8D3ASwD8EcRWWfdtto57YtzX+7lNUlfJREREREREflS0juGRURg3qyeBPArAB+awbb5zrZlAF6rqr9zLs8DcCuAKwF8AMAXk7xsIiKijKKqn4266EkA7xKRQQAfhDny5vUpeNwLYl3udBJvSPbjERERERERUWqkIkrivTBZhlsQzjRM1OUAzobJR/yde6GqBkXkIzCF4XeJyJdUVZO0XiIiynAtR09hUU0pGqtKvF6KH3wPpjB8mXWZ2xFcffrNIy7vTdGaiIg8paow/Ss0E8Pjk2jtGcHKhgrk5fH5y2SBIN8+k7fy+Tdk1rgPmz0+d9NLamFYRM6G6eb9hqo+ICIzLQzHHY6jqgdFZD+AVTAZiM9NsQ7mHxIR5QhVxQdv240jJ4dxxeoGbH3NGiyqKfV6WV7qck7Lrcv2AbgQZh8asY8UkQIAywFMAjiYjgUSEaXTV/+8Dz9/5Aj+3wvOwD9ddobXy8koOw714Pqf7ERlcQGuvKAZW1+zxusl0Sxd9uX70N7LcT3knRed3Yjvv+UCFohn6GO/3IO7n+nAZ169Bq9et8jr5WSUAx0DeMfPHkVNWRF+/o+bUFlS6PWSfClpGcPOG8ufAzgK4BOzvJu4w3EcB5zTVbO8fyIiyjLbD/XgYNcQAkHFw8+dRHVpzu/wL3JO7SLvvc7py2Lc/jKYCKeHVXUslQsjIkq3jv5RfPu+Z9E3MoGv/nk/RicCXi8po+w62gsAGBib9HYhRJTx7n6mA9sPnvR6GRnl2c4B3LyzFd2D4/jWvQem34Ai/PyRIzh8chiPt/bijt3HvV6ObyWzY/jTANYDuERVZ/tRZFKG4zD/kIgod9yw/Wjo+9eevwjlxalISfIX5wido6o6FHX5MpjhrwDwC+uq2wF8CcC1IvItVX3UuX0JgP9wbvPdlC6aiMgD9+/rCn0/HgjiqWN9uGDpPA9XlFlajp4Kfb9+SY13C6E5EwHYqElesFNMDp8cxsVnereWTHO4ezj0ffspdvzPVJv1nLX3Dk9xy9yWlHfPIrIZpkv4v1T1b8m4TyIioumcHBzDn54Mf/p73eYlHq4mra4B8EEReQDAEQADAM4A8EoAJQDuBPBV98aq2i8i74ApEG8TkZsB9AB4DczROrcDuCWtPwERURrcv78r4nzL0V4WhhMUDCoedzqGAWDDklrvFkNz9tBHZ5rySJQc/33XfnzzHtPt2naKxbmZsJ+vofEABscmUZEDTTDJ0jkwGv6+nwdGxjPnf1FOhMTPYOIfPjXHu+NwHCIiStjtj7VhImDaENYtrsGaRfF2H1nnPpiC7noAz4fJE+4F8BBMrNPPo4e0qupvRORyAJ+EGeZaAuBZAP8K4Jsc6kpE2WYyEMSDByILw7usDlia2rNdg6EIibqKYjTX5nR+PxHNkv23gznXMxP9fHX2j6KivsKj1WSeDqsY3DHAwnA8yfiooQLhzN/RONP+figiP4QZSvf+Ke5rn3MaL0N4pXMaL4OYiIhyRDCouGlHOEbiTZtyplsYqno/gPtnsd1fAbwi+SsiIvKfx1t70T8amY2760ivN4vJQLuOhIvoG5bUcKo7Ec1KszUUuo1xCDMS/Xx19I9hBQvDCZkMBNE9GC4Gd/aPTnHr3JaMwvAYgB/HuW4DTDfTQzBF3+liJu6F6WR6GYAv2FeIyAqYgvERcGo6EVHO+9vBkzh80hxeVVlcgFetW+jxioiIyE+27es67bIT/aM43jeChdXsfp1Oix0jsZQxEkQ0O821ZaHvmZM7M6d1DA+wuJmo7sFx2MdDdrAwHNecC8POoLm3x7pORLbCFIb/T1V/ZF1eBmAJgGFVPWptcj+AZwBcJiKvUdXfObfPgxmaAwDf4+GuRER0ozV07vUbmlBWxLwtIiIKs/OFiwvyMDYZBGC6hl+5loXh6dixG+sX13i3ECLKaAuqSyACqAIdA6MYnwyiqCDP62VlhOiOYebkJi66iH5qeAJjkwEUF+R7tCL/8up/4yaYAvDP7AtVNQDgbQCGAdwuIjeKyBcBbAdwFYC/AvhamtdKREQ+0zUwhj8/dSJ0PoeGzhERUQK6BsbwRLsZX5KfJ7hm4+LQdS3MGZ5W38gEDnQOAgAK8gRrm2u8XRARZayigjwsqCoBYIrDx/vYNZyI4fFJ9AyNR1zGrtfEdcQooncxZzgm331Mo6rbAWwE8FsALwHwAZihc/8O4MWqyt8kEVGOu+2xVkwGzcEjG5bUYPWCKo9XREREfmIPnbtgSS0uW1kfOs8BdNPb3dob+v7shVUoLWKHFRHNXhNzhmcsVuwGB6glLlYRPVaxmJKTMRyXqm4FsDXG5dsAxJ1eoKpPA7g6VesiIqLMFQwqbt7RGjp/3ealHq6GiIj8yI6RuPyseqxfUhM6/2R7Pw8nnUZEjIT13BERzUZzbSkedQZaMmc4MW29MQrD7BhOWKxhcxxAF5vvOoaJiIim8tCz3TjaY4bOVZUU4FVrOXSOiIjCAkHFA3ZheFU95lcUY9l8MwBpPBDE08f6vVpeRthlD55bwsFzRDQ3TbV2x/CwhyvJHLE6qxmFkLjOGM9VrMuIhWEiIsow9tC5Ky9oRkkhO76IiCjsifY+nBqeAADUVxZjzSITN7TeKnDahU+KFAwqHrc6hlkYJqK5aq4tC30fqxOWThczSqJ/FKrqwWoyT+woCXYMx8LCMBERZYzO/lHc9UxH6PybOHSOiIiibNvXGfr+spX1EDEJdhusSATmDMd3sHsQ/aOTAID55UVYPK90mi2IiKbGjOGZi9VZPTwewODYpAeryTyx8oSZMRwbC8NERJQxbn20FQFn6NymZfNwZkOlxysiIiK/ic4Xdtkdw4+zYziuXUd6Q9+vX1IbKqwTEc1WsxUlwYzhxLTH6axmcTMxnQMxMoZjXEYsDBMRUYYIBBU3RQydY7cwERFFOjU0jsdbewEAeQJcemZd6LrVCypR6sQPtfeO8JDSOFparRiJpTXeLYSIssYiq2P4RP8oJgNBD1eTGezO6oXVJaHvOUBtehOBILoHx0+7nPv92FgYJiKijPDAga7QJ+e1ZYV42bkLPF4RERH5zYPPdsONX1y3uAa15UWh6wry87C2uTp0ftcRxknEEtExvJj5wkQ0dyWF+aivLAZgmj1OsEA3pdGJQGjQXH6eROy7OEBtet2D4eeoKD9c9uRzFxsLw0RElBEihs5t4NA5IiI6nZ0vvGVVw2nXb1gaLnS2OJ3FFNY/OoH9nQMATDFi3eLqabYgIkoMc4YTd8yKkVhQVRLRcc2u1+nZcRtnNFQgP89EIvUOT2B0IuDVsnyLhWEiIvK9E32juHdv+M3+GxkjQUREUYJBxQP7u0Pn7Xxh1/rFNaHv2TF8uj2tfaGO69ULKlFWVODtgogoazBnOHF2vnBTbSkaq8JREswYnp5dPF9YXYIGp1sdQKgTm8JYGCYiIt+7ZWd46NxFK+bhjPoKj1dERER+8/Tx/tDho/PKi7C26fRuV7tjeE97H8YnmXNp23U0XCxfv6TGu4UQUdZpqmXHcKLs56e5phSNVeHCZgcHqE3LzmFuqCyOKAyz4/p0LAwTEZGvTQaCuHlnOEbius1LPVwNERH51f37u0LfX7qyDnnOoaO2uopiLJlXBgAYnwzimeP9aVtfJrALwxuWMF+YiJKnubYs9H1777CHK/E/u6O6ubYUDZXhjuEudgxPy84SbqgqQYPVcc2c4dOxMExERL62bV8XjveZT3bnlRfhpWsaPV4RERH5UUS+cIwYCZfdCWsXQnOdqqLlaG/oPAvDRJRMzcwYTljbqXDh3ERJsGN4Juyu4Maq4sjnjx3Dp2FhmIiIfO3GHeFu4asvaEZxAYfOERFRpL6RCeyyipqXroxfGLYLnvY2ue5g9xD6RiYAmA9il84vm2YLIqLERWQM97IwPBX7+WmuLYvoeO3oH4W6YfAUk53D3FhZgsZKZjRPhYVhIiLyrfbekYgOsDdu4tA5IiI63V+f7Q5l0a9trkZdRXHc29qF4RZ2DIfYw/jWL66ByOlRHEREs2VnDB/rHUEwyOJmPHZHdVNNKSqLC1BaaJpjRieC6B+d9GppGSGyY7gkYnhfJzuGT8PCMBER+dYtO47Cfc34/DPnY1ldubcLIiIiX7p/Xzhf+PJV8buFAWD1wkqUFJq3QW2nRtDJw3IBAC2tvaHv7SF9RETJUFZUgHnlRQCAiYAy6zWO8clgqLApAiysKYGIoMGKQ+jifmtKXREZw8Wot547/rs7HQvDRETkS5OBIG55tDV0/rpNHDpHRESnU9WIwXNT5QsDQGF+HtY21YTO7zrSm6KVZZbojmEiomRrisgZ5gC6WE70jYYaYxoqi0MxeoxDSMz4ZBAnh8YBAHkCzC8vinruWFSPxsIwERH50j17O0MveuoqivDiczh0joiITrevYwAnnDd6VSUFWNdcM+0265eGb9PSyjiJwbFJ7O8YAGDeSK9jYZiIUoA5w9Nr6w0XzJtrw1nvDRyglpCuwXDRvK6iGAX5eRw+Nw0WhomIyJdu3G4NnbtwMYoKuMsiIqLT2TESl66qR0H+9PuL9YutnGF2DGNPa2+oQ+2sBVUoLy7wdkFElJUiO4ZZGI4lOl/Y1VjFjuFEROcLA0BtWREK801ufv/oJEYnAp6sza/4LpuIiHyntWcYDxwwb/RFgDdu5NA5IiKKbdsM8oVdG6yO4T3tvZgIBJO9rIyyyxrCt35JjXcLIaKsZncMszAcW7v1vNjPV2NETi67XuPptIrmDZXmOcvLE9RbQ2k7WViPwMIwERH5zs07j0KdzqVLV9ZjyfyyqTcgIqKcNDg2iUeP9ITOb0mwMNxQWRJ6wz06EcTe4wMpWV+m2HW0N/T9hiUcPEdEqdFkRSMwYzi2iI5hqzDcYOXksrAZn100b7C6rO3vO1hYj8DCMBER+cpEIIhbH20Lnb9uE7uFiYgotoef7cZEwHySePbCqog3ftNZbxVA7Y7ZXKOqaLF+/g3sGCaiFGHG8PTamTE8J5FREsUxv+fzF4mFYSIi8pW7n+5A14D5FLyhshhXnN3g8YqIiMiv7t8/8xgJl10AzeXC8OGTwzg1PAEAqCkrxPK6co9XRETZyu6AbT81AnUPEaSQhDKG2fEal52/bD9nzGiOj4VhIiLylRt3hIfOXbNxMQoTGCJERES5R1Uj8oW3nDXTwrA1gM6KUsg1u45Y+cKLayAiHq7GH0TkSyJyj4i0isiIiPSISIuIfEZE5sfZJl9E3i4iD4jIKWe7gyJyi4isSvfPQORHVSWFqCoxwy3HJoPoHhz3eEX+MhkI4kRfuOgbmTEcGSXBonpsnQOnZwxHf9/JjuEIfLdNRES+ceTkEB480A3ADJ27ZuNij1dERER+9VzXUOhQ5IriAlywdGbZuGcvrEJxgXk7dLRnGN2DudlB1NJqx0gwX9jxAQDlAO4C8A0ANwCYBLAVwB4RiXiBIiIVAP4C4IcAKgH8n7PdXwFsBsDCMJGDOcPxdQyMYTJoCr51FUUoKcwPXVdRXICyInN+bDKI/pFJT9bod50RURKxM4bt4jEBBV4vgIiIyHXTjtbQ91tW1UfkahEREdm27esMff/8M+fP+AiTooI8nNdUjUedjtldR07hJWsWJHWNmWDXkd7Q9+tZGHZVqeppLWUi8nkAnwDwcQD/Yl31fQAvBPAuVf1+jO0KU7VQokzTXFuKZ473AzA5w/y7E9YeMXju9PdBjVUlONQ9BMDESVSX8U9LNDs/uCEiY7gk5m2IHcNEROQT45NB3P5YuDB83ealHq6GiIj8LjJfeHZ59OutnOGW1t45rijzDI1NYu8JU6ARAdYtrvZ4Rf4QqyjsuNU5XeleICIbAFwH4JZYRWHn/iaSu0KizGXn5tqFUIoaPGc9Ty47DoHFzdONTQZCmfn5eYL55Rw+lwh2DBMRkS/85ekToZyxBVUleMEMsyKJiCh3jIwHsP1QT+j85bPcZ5johEMAIrN2c8Wetj44Ry3jrMZKVJaw+2war3ZO91iXXeec3iQi1c5tFgM4CeBeVX020TsXkcfiXLV6pgsl8is7N7eNheEIbT3h58N+nlwcoDa1Tus5qasoQn5eODO/obIk5u2IhWEiIvKJG7dHDp0r4NA5IiKK45GDJzE+GQQArGyoiOhAm4kNVi7xnrY+TAaCObX/2XXUGjxndU+TISIfAlABoBrAhQAugSkKf9G62UbndCmA5wDYw+lURL4L4L2qGkj9ion8zy54ujnxZNjPR1OMwnDEALUBdr1Gs7OD7SI6ANSWFaIwXzARUAyMTWJ4fBJlRSyJAiwMExGRDxzqHsLDz50EAOQJcO0mDp0jIqL47HzhLXM4wqSxqgSLqktwrG8UIxMB7D0xgHObcidOoSWiMMyczxg+BKDROv8nANerapd1mZtj8t8AfgPg3wC0wQyd+x5MFnEXzOC6KanqBbEudzqJN8xs6UT+1FTD4XPx2B3U03UMs+v1dPbgObtDGABEBA2VJaHie2f/GJbVsSQKMGOYiIh84KYd4W7hF65uwMLq2XV+ERFlo2BQcd++TuzOwQzceJKRL+xab3UN24XSbKeqaDnaGzq/gYXh06jqAlUVAAsAvAHACgAtTq6wy31PvRfANaq6V1UHVfUeAFcBCAL4VxEpSufaifwqomP41AhU1cPV+EtEx3DN6cPnGpiTOyX7ObEzhWNdxucvjIVhIiLy1OhEALc9ag+dW+LhajKDiMwXkbeLyK9F5FkRGRGRPhF5SET+UUTyom6/TER0iq+bvfpZiGh6N2w/grf9ZCde/52/4tHDPdNvkOUOdw/h8EnTZVZamI+Ny+dW0LQLorusQmm2O9ozjJNDJtu/qqQAK+rKPV6Rf6lqh6r+GsBLYKIifmZd3euc3hEdF6Gqu2FCrCsBnJ2GpRL5Xk1ZIcqK8gEAQ+MB9A5zNiNgPgS2h/HFipKIzBhmYTNahxUlEd0xDEQ9fwPsuHaxb5qIiDz156dOhKbHNtWUzrnzK0dcDeC7AI4DuA/AUZhDXd8A4EcAXi4iV+vpLRi7YQ5zjfZk6pZKRHN1x+7jAICgAj9+6BAuXDbP4xV5y+4WvviM+SguyJ/T/dnZurnUMbwrKkYizxrSQ7Gp6hEReRrA+SJSp6rdAPYB2IRwgTia+0TzcCgimEP6m2tLsb9jEIDpkq0tZ0N99+AYxgMmO7+mrBAVxaeX6yIzhlnYjGbHa8TqGI54/lhYD2FhmIiIPHVD1NC5fL4xTcR+AK8B8AdVDboXisgnAOwAcCVMkfiXUds9rqpb07VIIpq7kfEAHrciJO56ugNdA2Oorzz9DU+usAvDc8kXdq1ZVIWi/DyMB4I4fHIYJwfHML8i+59fxkjM2iLn1O0OvhvAWwCcG31DESkGsNI5ezjlKyPKEE014cJw26nhnMp2j6d1mnxhAGiIyhhWVYjwvZPLHsgXPXwOiHr+WFgPYZQEERF55tnOAew4ZA6Lzs8TXLORQ+cSoar3quoddlHYufwEzKAbANiS9oURUdK1HD0V6iACgMmg4le72jxckbdGJwJ4+Lnu0PlkHGVSXJCPc5uqQudbciROIrJjuMa7hfiMiKwSkdOqVCKSJyKfhxk297Cquk/gLwEcA3CNiGyK2uxTAKoB3Ofso4kIQHOtPYBuZIpb5o7IfOHYheGK4oJQJ/F4IMgYjih2vEZDzIxhRnHEwsIwERF55sbt4WzhK1Y3xPxkl2bMfYU4GeO6RSLyThH5hHO6Np0LI6KZe+TgydMuu2Vna84O69l5uAejE6ZQvqKuHEvmnz6cZzbWWx2zLa3ZHycxPD6JZ44PAABEgPNZGLa9AsAJEblLRH4gIl8Qkf8FcADAJwCcAPAO98aqOgTgegAK4EERuUlEvioiDwL4JIBOAO9M9w9B5Gd2fi4Lw0bbqeHQ93bhPFrEALoBFjdtHRFRErEyhjl8LpakREmIyJcAXAhgFYA6ACMAjsDkGH5bVU9/RRv7fg4DWBrn6g5VXTDnxRIRkS+MTgTwS6vrjUPn5k5ECgD8vXP2TzFu8mLny95mG4C3qurRGLeP9RiPxblqdYLLJKIZeOTQ6cPmDnYPYfuhHly0Yr4HK/LWtn3hGInLVs09RsK1YUktfoxDAIBdR3qTdr9+9URbHwJB8+HCyoYKVJUUerwiX7kbwJkALgGwHkANgCGYGKefA/imqkb8x1TVu5xu4U8BeBFMl7B7FM/nVPVY2lZPlAHsqAS7UzaXRQyei9MxDACNlSU42DUEwMRJrGaVDIB5b9k3YvpjCvIE88pOz622B9IxSiIsWRnDHwCwC8BdMJ+IlgO4CMBWAP8kIhepamv8zSP0Afh6jMsH575MIiLyizufOB7aeTfXluKylcl7g5/DvgiTcXinqv7ZunwYwOdgPrA96Fy2FmY//QIA94jI+U7XExH5xOhEAI9bsQYvXdOIPz/VAQC4ecfRnCwMJztf2LVhaU3o+91tvZgMBFGQn70HV+6y/l2tX8x8YZuqPgng3bPYbjeAq5K/IqLsYxc+2TFs2M9DU5yMYSCqY5hdryFdVqG3vrI45kBVu2PYHlSX65JVGK5S1dP+RToZTJ8A8HEA/5LgffVyMA4RUfa70Ro698ZNSzgNfY5E5L0APghgL8wQnBBV7QTw6ahNHhCRlwB4CMBmAG8H8I3pHkdVL4jz+I8B2DDzlRNRPLusfOEz6svxnheuDBWG73zyBLYOj6MmRkdMtmo7NYxnO02vSHFBXlIL4wurS7GgqgQn+kcxPB7A/o5BnLOoavoNM5SdL2wXxYmI0sGOSmi3IhRymd05HW/4HBAZkcCu17DIfOHY8YTVpYUoKsjD+GQQg2OTGBybDGU257KkfAweqyjsuNU5XRnneiIiykH7Owbw6BHzprQgT3D1hc0eryizici7YYq6TwN4QfQhrvGo6iSAHzlnL0vR8oholh45GP6vfNGK+Ti3qTo0JG18Mohft7R7tTRP2N3Cm1fMR0lhflLv3y6Q2oXTbKOqEQP2NixhxzARpVddRRGKC0w5qn90Ev2juT1ETVUjM4ZrpsgYrmTHcCwR+cKVpw+eAwARieoa5vMHpH743Kud0z0z2KZYRN7sDMZ5n4i8QESS+6qPiIg8ZXcLv2RNY0TeE82MiLwfwLcAPAlTFJ7p1HO30lKezHUR0dxttwbPud2x124M57HfvCO3htDZ+cJbkpgv7LILpNlcGG47NYLuQfMGurKkAGfUV3i8IiLKNSISEZfQnuNxEj1D46HBqpXFBagqjd/FGtExzDiEkM4Bu2M4dmEYYM5wLEntmRaRDwGogAnbvxAmsH8PTOZhohbAhPrbDonI21T1/gTXwcE4REQ+NTIeNXRuU7yZozQdEfkozD72cQAvVtXuWdzNRc7pwSlvRURpNToRQEtrb+j85hXzAACvPX8RPv+HZzAyEcC+jgG0tPbmRMfn+GQQDz8b/hN3eRLzhV3rl9SEvreznbONXfQ+f3ENo5yIyBNNNaWhIWptp0Zw9sLsje+ZTnS+sEj8v8sRHcMD7Hh1RXYMx286amRG82mS3TH8IQCfAfB+mKLwnwC8RFW7ptrI8hMAV8AUh8sBnAfg+wCWAfijiKxL8nqJiCjNfr/nGAZGJwEAS+eX4eIzcm94UjKIyKdgisKPAbhiqqKwiGwQkdP2+SJyBcwAWQD4RUoWSkSz0nK0F+OT4Xxht8OlsqQQr1q7MHS7W3YkOt85sz16pAdD4wEAwOJ5pVhRl/yDHNYsqkZhvnkzfrB7CKeGxpP+GH7AGAki8gPmDIclmi8MsGM4HjsWojFOxjAQ1THM5w9AkjuGVXUBAIhII4CLYd6wtojIq1R1VwLbfzbqoicBvEtEBmEG6mwF8PoE7oeDcYiIfOrGHRw6N1ci8lYA/w4gAOBBAO+N0VlwWFV/6nz/3wBWisjDANx27bUAXuh8/ylVfTiliyaiGXnEipHYHDVk7dpNS3DbY+a/8h17juFTrz4n64en2PnCl6+qn7KbarZKCvOxZlE1Hnc6tVtaT+GFqxuT/jheszuG7S5pIqJ0sgugbTkeJWHnCzfVTF0YtmMSOgdGoaop2Sdmmo4EoyTsojE7ho2UvIJU1Q4AvxaRXQD2A/gZgHPncJffgykMczAOEVEGe+Z4f6hTqTBfcNUFHDo3S8ud03yYo3RiuR/AT53vfw7zwepGAC8HUAigA2ZI7LdV9cFULZSIZmf7odPzhV0bltRgVWMF9ncMYng8gN89fgzXbV4SfRdZ5f6IfOGGlD3O+iU14cLw0d6sKwyPTgTw9LH+0Pn1i9kxTETesAvDdsdsLrIzlu1O6ljKigpQWVKAgdFJTAQUp4YnMK+8KNVL9D27+3eq+TWRURzsGAZSPHxOVY/ATEhfIyJ1c7grDsYhIsoC9tC5l65ZgLqK+J/mUnyqulVVZZqvLdbtf6yqr1LVZapaoarFqrpEVa9hUZjIf0YnAthlHe5/0fJ5EdeLCK6xh9DtPIpsdqJvFHtPDAAAivLz8LwURhBl+wC6J9r7MBk0AwvPqC9HdVmhxysiolxld8ayYzgyY3g6EcVNdr0CiHweGhPsGO7kcwcgxYVhxyLnNDCH++BgHCKiDDc8PonftLSHzmd7dxsR0Ww93hrOF15RX46GGFl5b1jfhKJ881J+T1sfnjrWl9Y1ptMDVozExuW1KE9hbMaGpeHC8ONHexFwiqjZYteRcLGb+cJE5KWIjOFc7xieQcYwwDiEaCPjAfQ7M2wK8wW1ZfE7qBsjojjYMQwkoTAsIqtEpDrG5Xki8nkADQAeVtVTzuWFIrJaRM6Iuv3ZInJaR7CILAPwbecsB+MQEWWoO3Yfw8CY2WGvqCvH81Zw6BwRUSwR+cLLY/+trC0vwsvOXRA6f3MWD6Hbtr8z9P3lq+pT+liLqktCnVhD4wEc6BxI6eOlm90FbRfBiYjSraGyODTws2doHMPjkx6vyBuqGtkxPE3GMMABdNE67XzhypIpZ9g0RBXVVbPrA+DZSEbH8CsAnBCRu0TkByLyBRH5XwAHAHwCwAkA77Bu3wTgGQD3RN3PNc79/EFEviMiXxKR253bngngTgBfTcJ6iYjIA3aMxBs3LeGQBCKiOOzC8EUr5sW93bWbFoe+/83j7RgZn8sBev40GQjiwQPdofNbzkpdvjBgYjoi4iSO9Kb08dJJVSMiSjh4joi8lJcnWGQVQdtzNE6if2QSg07zTGlhfkJ5wfZwNXYMAx1Wcby+cuqowqqSApQUmlLo8Hgg9NznsmQUhu8G8GMA9QDeAODDAK4E0APgswDWqOrTCdzPfQB+D+AMANcB+FcAlwN4CMBbAbxKVceTsF4iIkqzJ9v7sLvNHOZclJ+HKzl0jogoptGJQGhIJ3D64Dnb81bMx7L55lDcgdFJ/OGJ46leXtq1tPZiwDk8dGF1CVY2VKT8MTcsrQl9n005w+29I+hyDputKC7AyoZKj1dERLmOOcNA66nh0PdNtaUJNc/Yw9UYhxDZMTxVvjBgPgDm8xdpzgFdqvokgHfP4PaHAZz2L11V74eZoE5ERFnmBqtb+OXnLeDkXCKiOHa39mLMzReuK484XDSaO4TuS3/aCwC4ZedRXJVlH7zdvy+cL7zlrPq0HG2y3uoYbsmiwrDdLXz+4hrkT3GoLRFROth5um05mjM803xhILL4yY7hyI7hqV43hW9TjKM9w862ozijPvUfOvtZOobPERFRDhscm8TvHreGzm3i0DkiongeOdgT+n7zFDESrqsuaEaBU+DbefgUns2yTNx05gu7zmuqDj2nz3UNoXc4Ow5atIvcjJEgIj9oqgkPoGuzOmdzyUzzhYGo4XPseEVnv90xPH1huIEZzRFYGCYiopT67ePtGHJyL89sqMCm5dMXOoiIclVkvvD0QzrrK4vxorMbQ+ezaQhd18AYnmzvBwAU5AkuPrMuLY9bUpiPNYuqQucfb+1Ny+Ommt0xbOcoExF5xe6QzdWMYfvnbq4tm+KWYY12FAI7hiO6pqfLGAYinz92XLMwTEREKaSqHDpHRJSgsclARKbt5uXTF4aByCF0v9zVhrHJ7BhC98D+cIzEhqW1qCopTNtj23ESdkE1U41OBPD0sb7Q+fMX13i3GCIiR1MtM4bbojKGE2EPn+saGEMwqElfVyaxc4IT6xgOP3/MGGZhmIiIUmhPWx+eOma6vYoK8nDlhiaPV0RE5F+7W/tC+cLL68qxoHr6NzcAcOnK+tDhp6eGJ/CXpzpStsZ0un9/ZL5wOtlRC9mQM/zUsT5MBEzhYEVdOWqZ9U9EPhDRMcyM4YQzhksK81FVYkaGTQYVPVkSeTRbHf2JD5+Lvg07hlkYJiKiFLK7hV+1diFqyvhGlIgoHjtGYvMMYnfy8wRXXxgeOnfzzqNT3DozBIKKBw6EC8Ppyhd22VELjx/tzfhurF1HekPfr2eMBBH5xIKqktAgzK6BMYxOZMcRLzNhd0o3J5gxDETlDOd4cdPOCbZjIuKJjOJgxzALw0RElBL9oxP43e5jofNv2syhc0REU5lpvrDt7y5cDOe9Nf767EkcOTmUzKWl3Z62XvQOTwAweYHnLKyaZovkaq4tRV2F6SgaGJvEs12DaX38ZLMjSjYsrfFuIUREloL8PCywCpzHcqxreGB0An0jZl9XVJAX2u8kopED1AAAw+OTGBibBAAU5eehpmz62KmGiOF9uV1UB1gYJiKiFPltSztGnE/9z2qs5KAbIqIpnJYvvGJmgzoX1ZRGdNXesjOzh9Bt2xfZLZzufHoRwQYrTmLXkcyOk2ixcpLXL+b+mIj8I5dzhu0YiaaaUuTlJb6vi8zJzd3ipl0Ur68sTuj1QsRz1z8G1cw+KmiuWBgmIqKkU1XcYMVIXLeZQ+eIiKayp60PoxMmX3jZ/DIsrE78cFLXtZvCR2bc9lgbJgPBpK0v3ex84XTHSLg2LA0XUFsyeADdsd4RnHAOMy4vysdZCyo9XhERUVgu5wy3n5p5vrCrodKOksjdjuGZ5gsDQGVxAUoL8wEAIxOBUMdxrmJhmIiIkq6ltRd7TwwAAEoK8/C69Rw6R0Q0lUees/OFZxYj4Xrh6gbUV5o3RV0DY7h3b2dS1pZuPUPj2N3WCwDIE+DSlXWerGP94prQ97syeACdvfZ1i2tCeZ5ERH5g5+q2nRr2cCXpZ3dIN80gXxjgADVXx4CVL1yV2NBeEYl4/jpz+PkDWBgmIqIUsIfOvXrtIlSXTp/1RESUyx45ZOULnzGzGAlXYX4err7AHkKXmXESDx7ogntU5/mLazwbXLq2OVxEPdA5GMqBzDQRMRJWPAYRkR8015aFvm/P4SiJmXYMRw6fy92O4c6IjuHECsNAVM5wDj9/AAvDRESUZH0jE/j9nvDQues4dI6IaErjk0E8ZmXYzrZjGACu2bg49P22fZ043pd5b7LtGIktZzV4to7SonycvTAcu7C7tdeztcxFxOA55v0Tkc/kcsaw3SHdNOPCcLjjtSuXM4YHIjOGE9VQyY5rFwvDRESUVL/e1RbKyTx7YRXOtw7FJSKi0+1p6w393Vw6vwyLZng4qW3p/HI8/0xTWA4qcOvOtqSsMV2CQcUDPsgXdtmF1EyMkxibDOCp9v7Q+fUsDBORzzBj2LA7pxPBjGGjY5Ydw/Zt7eJyLmJhmIiIkkZVceMODp0jIpqJRw7a+cKzi5GwXbMxfKTGrY+2IhDMnGnbTx/vR/fgOABgXnkRzmuq9nQ9kYXhXu8WMktPHevHeCA81HBeuTexHERE8SysLoX7duFE/yjGJzN3cOpMzSVj2O6O7Rocy6h9fTLNZvhc9G3ZMUxERJQkjx05hf0dgwCAsqJ8vO78RR6viIjI/x452BP6/qIVs4+RcL10TSNqy0y2e3vvCB480DXNFv6xbV94YN5lK+uQ5/GgNDuT9/GjpxDMsDfeu44wRoKI/K2oIA+NTverKnCiLzeKdCPjAZwcMh+EFuTJjLpdAaCkMB81zr4+EFScHMrNrtfO/pkPn4u+bWcOd1wDLAwTEVES2UPnXrNuESpLOHSOiGgqp+ULJ6EwXFyQjzdssIbQ7cicIXR+yRd2LZlXhvlOl23/6CQOdg96vKKZiRg8t5SFYSLyp8ic4eEpbpk92nvDP+fCmpLQsNOZaKxkcdPu9m2YQcZwPTOGQ1gYJiKipOgdHsfvnzgeOs+hc0RE03uivRcjEwEApgg500NJ43njpvAQuruf6UBXBuTn9Y1MhOIaRIBLV9Z5uyAAIhKRy7vrSK93i5mFFisXeT0z/4nIp+yc4bYcyRm2YySaa2aWL+xqsOIQOnNwAN3g2CSGxs1rqKKCPFSXJt6UxIzhMBaGiYgoKX65qz2UCXZuUxXWNtd4uyAiogxgx0gkI1/YdWZDJS50OkQng4pf7vL/ELq/Ptsdykhc21SN+RWJd/6kkh0n0dKaOQPoTvSN4phzSHZZUT5WL6j0eEVERLHZH4raA9mymT1oz+6YnolcH0DXGZUvPJPZNnZhuKN/FKqZFRWVTCwMExHRnKkqbtx+JHT+uk1LPVwNEVHmsAfPJSNf2HbtpvCRG7fsbPX9mx47X/jyVfUeriTShgztGN5ldQuvba5GQT7f+hGRPzXXhjtm23KkMBzRMTzLwnCuD1Czi+F2rEYiKooLUF6UDwAYmwyif2QyqWvLJHx1QEREc7b9UA+e6xoCAJQX5eM1HDpHRDStiUAQjx6284WT1zEMAK88byEqSwoAAIe6hyK6k/1GVSPyhS/3Qb6wa93iarjRj/s7B9A/OuHtghIUESPBwXNE5GN2x6ydvZvN7M7o2cZIRXa95mDHsBWfYcdqJCri+cvBKA4XC8NERDRn9tC5165vQkVxgYerISLKDHva+kL5wovnlUZ0TCVDaVE+Xnd+U+j8zTuPTnFrb+09MRB6U1tdWojzfZSHW1ZUgNULqgAAqsCe1j6PV5SYXdbguQ0sDBORj9mF0dzpGA4XwGe7/7c7hrtysLBpD9xrmGHHMBA5gC5Xh/cBLAwTEdEc9QyN409Pngidv24Th84RESXCjpHYvDy5MRKua60hdH988gR6h8dT8jhzZXcLX7qyblbT2VNpw9Ka0Pd2RINfjU8G8UR7uIBt5yQTEfmNHaVwom8Uk4Ggh6tJDztjeLZREvU5njHcEZExPPPCcHTOcK5iYZiIiObk9sdaMe68eFu3uAbnNlV7vCIiosyQynxh15pF1TjP+bs8PhnEr3a1p+Rx5sqv+cKuiJzhDCgMP328PzQQdsm8MtT5ZJAfEVEsJYX5qKsoAmAGpnYMZHeRc2wyECrk5gmwoHrmRU2AGcP2v5PGWUVJWM9fDnZcu1gYJiKiWVNV3LSjNXT+TewWJiJKyEQgiMeOWPnCy5ObL2yzu4Zv3nnUd0PoBscmI7KW/VgYtjN6W472+u45jLbL+re1gd3CRJQBmqw4hfYsj5M43hsuQi6oKkHhLIeD2lEI3YNjCAT9vW9KtmR2DDNKgoiIaBb+9txJHOo2Q+cqiwvwqnULPV4REVFmeKK9D8PjJl+4ubYUi+clN1/Y9pp1i1BaaCZv7+8YREtrb8oeazYefrYbk86b2XMWVqFhFm/uUm3Z/DLMKzfdbH0jEzjo7Pv8yu5q3rCU+cJE5H/NETnD2T2Azs5Rnst8geKC/NC+KajAycHcKm52DdgZwzPvGI7IGGbHMBER0czdsCM8yOj1G5pQVsShc0REiUhHvrCrsqQQr7Y+uLt5h7+G0G2z8oUvP8t/3cIAICJYbw3Eszty/ajFGjy3fjELw0Tkf3bObrZ3DLf3hgvfTbPMF3bZBdFcyhlW1YiO4dl8qByZMZw7z100FoaJiGhWugfH8JenrKFzmxkjQUSUqEcO9oS+v2hF6mIkXNdaUT937D6OgdGJlD9mIlQV9+8LF4a3+DBGwmUPcPNb17Wts380NNSopDAPqxdWerwiIqLp2QXStiwvDEd2DM+xMJyjA9QGxyZDR16VFOahqmTmDUocPmewMExERLNy26NtmAiYQ383LKnB6gVVHq+IiCgzTASCeOywXRhObccwAKxfXIOzGk2BcGQigN/tPpbyx0zEc12DoSJmZXGBr2MPIgbQ+bhjeJfVLby2uWbW2ZVEROkU0THcm92FYbsjuqlmboXhxsrcHKBmd/g2VpVARGZ8H3a3dWf/mO/nB6QKXyUQEdGMBYOKm6xDka/bvNTD1RARZZYn2/sw5HS5NNWkNl/YJSKRQ+iswaFe2mZ1Cz//zDpfFzHXLq5BnvO+c3/HAAbHJr1dUBwtVr7weg6eI6IM0VQT3hcyYzhxuTpAzc4Enk2+MACUFxegoth0Go8Hgugd9sfRVOnm31deRETkW399rhtHe8wLtqqSArxqLYfOERElyo6R2JyGGAnX69c3oajAvPx/or0PT7b3pe2x47k/A/KFXRXFBVjldF0HFdjj0ziJiMFzS/zbgU1EZLOjJI71jiIYzN7uTbsjes4Zw1W5OUDNLoLPZWht5POXO4V1GwvDREQ0YzduD3cLv2FDM0qcafdERDQ9e/BcOmIkXDVlRXj5uQtC52/e6e0QuuHxSWy3iuSX+zhf2GVHXdgFWL+YCASxpy1c8GfHMBFlioriAtSUFQIw3Ztdg9lZpJsIBHG8L1wYXlg9+6ImADRU5uYANTsTuLFy9s9hYyVzhlkYJiKiGekcGMVdT3eEzr+JQ+cyiog0i8j/isgxERkTkcMi8nURYVsZURpMBoJ41MoXfl4aC8MAcO3G8N/s37Ycw/C4d3EIjxw8ifFAEACwqrECi+aYs5gOETnDVpavXzxzvB9jk+Y5ba4tjSgYEBH5XXMODKA70TcKtxm6obJ4zg02jVbHay4VNiMzhmcXJRG9bS49fzYWhomIaEZue7QNk86rmY3LarGykdPOM4WInAHgMQBvA7ADwNcAHATwPgB/E5H0VqiIctCTx/oj8oXnOo18pi5aMQ/L68oBAANjk7jziRNpfXzb/Va+8JazGjxbx0zYHbgtR0/5blCNPRSPMRJElGnsQWzZmjNsF7znGiMBRGUM51AUQkTG8BwKww05+vzZWBgmIqKEnT50jt3CGeY7ABoAvFdVX6eqH1PVF8IUiM8C8HlPV0eUA+wYic3L581qivZciAiu2WgPofMuTmKbnS+cATESALCirjx0qPOp4QkcPumvwkWLlXu8gTESRJRh7EFsdg5vNrF/rrkOngOAuopwUbR7cAyTzpE42c7OGJ5LlIQ9uK6THcNERERTe+BAV+hT7pqyQrz8XA6dyxROt/BLABwG8D9RV38GwBCAt4hIeZqXRpRTvMoXtl25oRkFeaYg/eiRUzjQMZD2NRzuHsIRp6haVpSPC5dlRneriGD94prQebtD1w/s3OP17BgmogwT2TGcnYVhuxO6KQkRSkUFeZhfXgQAUAW6B8fnfJ+ZoCOiY3gOGcNVuZnRbGNhmIiIEmYPnbuSQ+cyzQuc07+oakQrgaoOAPgrgDIAF6V7YUS5wuQLhwt3XhWG6yuL8eJzGkPnb97ZmvY1bNvXGfr+4jPmo7ggc/YndsG1pdU/heGugTG09phCSnFBHs5eWOXxioiIZsaOV2rP0sKw/XMlK06qoSq3BqipauTwuTllDFvP3UD2P3exJKUwLCJfEpF7RKRVREZEpEdEWkTkMzPNK+RQHCIif+roH8U9e8Nv5N+4iTESGeYs53R/nOsPOKerproTEXks1heA1claKFG2eupYPwbHzLC3RdUlWDzPu2Fr11p/w3+1qw1jk4G0Pv79doxEhuQLuyIG0B3p9W4hUVqsbuG1zdUoKmAPEBFllqZaZgzPRq4NUOsfncTohOlzKS3MR0Vxwazvy37uOtkxPCcfAFAO4C4A3wBwA4BJAFsB7BGRxfE3DeNQHCIi/7plZysCztC5zcvn4cyGCo9XRDNU7Zz2xbnevbwm9Ushyk0R+cIr5qc9X9h26Zl1oUNYTw1P4M9PdaTtsUcnAvib9VxsyZB8Yde6xdVwf3V7T/RjyCn2e23X0d7Q94yRIKJMFJ0x7LcBn8lgZwwvTlZhuDK3Bqh1DUR2C8/l9VRDxHM3mpX/5qaTrMJwlapepKr/4AyyeY+qbgTwnwAWAfh4gvfDoThERD4UCGrEgCIOnctdqnpBrC8Ae71eG5HfReYLz/NwJUBenndD6HYc6gl1+qyoL8fieXMfvpNOlSWFWNVQCQAIKrCnLd7nbell5wtz8BwRZaLq0kJUOt2foxNBnBzKrrzcQFBxzCoML0pCxjAANFTl1gA1Owt4LvnCAFBalI/KEvNvbiKgODU8Maf7y0Sz77e2qGq8f3m3AvgEgJXT3UcCQ3H+CWYozgdVdWj2qyUi8qfuwTF0D/rzE97Hj/biWJ/5Uz+vvAgvO3eBxyuiWXArF9Vxrncv7039Uohyj1/yhW1XX9iMr9+9H0EFHn7uJI6cHMLS+amfPxkRI5Fh3cKuDUtrsM8Z2rfr6Ck87wxvf5+TgSD2tPWGzrNjmIgyVVNtKfaeMH9f20+NoK5i9vmxftM5MIpJ5wjM+eVFKCtKSkkuKmPYn+8nkykyX3huhWH3PgZGB0P3Pc8Z5pcrkvOvML5XO6d7ErjtlENxROSvMIXjiwDcM9UdOVmHsTD/kIh86Ve72vDB23YjE45cufqC5owaEkQh+5zTeBnC7oe48TKIiWgOnj7ejwEncmBhdQmW+KBLdmF1Kbac1YB7nfz4W3a24iMvS/3LZXvwXKYWhtcvqcVNO8zQPjvb1yt7TwyEurCbakqT8kaZiMgLzVZhuO3UCNYtrvF2QUmUinxhAGistDKGc2CAWkTHcOXcPzhorCrGs53hwnCuDW9N6kQCEfmQiGwVka+JyIMAPgdTFP5iApsnZSgOEVGmCQQV//WX/RlRFBaJHFhEGeU+5/QlIhKx/xeRSgDPBzAM4JF0L4woF0TkCy+f52m+sO1aK07itsfaMBEITnHruWvtGcZzXebgv+KCPF90Ts+GHdXQcrTX80xCO0ZiPWMkiCiDReYMZ9cAunarMNyczMKw9WFgLgxQ64zKGJ6rhhzLaI6W7I7hDwFotM7/CcD1qtoV5/a2pA3FcbIOT+N0Em9IYC1ERGnzwP6u0BCC4oI8LEvDYbyzUVgguHbjEiyv8+f6aGqq+pyI/AXm6Jv/B+Bb1tWfhRki+33GNRGlxiMHe0Lf+6kY+sLVDWioLEbnwBi6BsZw795OvHRN6uKC7BiJi1bMR0lhZh6BsqKuAlUlBegfncTJoXEc7RlOSwxHPC3W4LkNjJEgogzWZOXu2h222aDtVLjQ3ZSkfGEgKmM4BzqG7eJ3Mo6QybWM5mhJLQyr6gIAEJFGABfDdAq3iMirVHVXMh+LiChb3LA9PPDn75+3FJ985Tkeroay3L8AeBjAN0XkCgDPANgME+e0H8AnPVwbUdYKBBU7D/mzMFyQn4erL2zG/9z3HAAzhC5dheEtZ2VmjARghvetX1Ib+nl2HT3laWGYHcNElC3sTtr2LCsMt/faHcPJi5SqqyiGCKAKdA+OYyIQRGF+UgMCfMXOGLa7fWersTK3MpqjpeRfiqp2qOqvYbqS5gP4WQKbcSgOEeWc430juHdvR+j8GxnTQCmkqs8BuBDAT2EKwh8EcAaAbwC4SFVPxt+aiGbr6WPhfOEFVSVYOt/7fGHbNReG9z337++KmJieTOOTQTz8bHfofKbmC7vWR8VJeKV7cAxHTpoutKKCPKxZFO/tFCVKRL4kIveISKuIjIhIj4i0iMhnRGTaT3ZE5Ecios7XmelYM1G2sLN3s69j2MoYTmLHcGF+HuaXh7teu7I8DqEjyVESjRHD+3KvYzilHyGo6hEATwNYIyJ109ycQ3GIKOfcsrMVzmBaPG/FfKyor/B2QZT1VLVVVd+mqgtVtUhVl6rq+1XV++lJRFkqIl94hX/yhV1L5pfhkjPNS/WgArc+2pqSx3n0SA+GxgPmMeeVZXw0kR3ZsMvDAXSPW0Xp85qqUVSQvV1iafQBmIilu2A+PL0BwCSArQD2iMjieBuKyKsB/COAwdQvkyj7RGYMj3ie4Z5MERnD85JXGAYiC6TZnJOrqhFREg3JjpLI4ucunnS8aljknAamuR2H4hBRTpkMBHHLzvCb7+s2s1uYiCgb2YVhP8VI2K7dFK5z3bqzFYFg8t+I378vHCNx+ap63xXIZ2rd4prQ988cH8Dw+KQn64iIkbDWRHNSpaoXqeo/qOrHVPU9qroRwH/CvL/9eKyNRKQewA8B3ALgsfQtlyh71JYVotTJnx8cm0TfyITHK0qOYFDR1puajmEAaKgMFzezueu1f2QSY5NmUG55UT4qiueekGtHSeRixvCcC8MiskpETjteSUTyROTzABoAPOx2IolIoYisFpEz7Ns7h7f+BcAymKE4Nncozs85FIeIssW2fV043md2PPPLi1Ka6UhERN4IBBU7fJovbHvxOY2YV14EADjWN4oHDiQyO3pmsiVf2FVdWoiVDeZIn0BQ8URbvBnaqWUXhjcs5eC5ZFDVeJWBW53TlXGu/4FzGv1+logSJCIROcPZEifRPTSGcaegWV1aiMqSwqTevx2HkM3FzcgYibl3CwOndwwHU/DhuJ8lo2P4FQBOiMhdIvIDEfmCiPwvgAMAPgHgBIB3WLdvghl2c0+M+/oXAJ0wQ3F+49zXvTCH8nAoDhFllRt3hIfOXXVhMw/9JCLKQs8cD+cLN1YVY5nP8oVdxQX5eMP6ptD5W3YkN07ieN8I9p4YAAAU5efheWf4s0A+U5FxEr1pf/zJQBB7rIK0vR5KiVc7p3uirxCR6wG8DsA7mdlPNDfZmDOcqnxhV0NVbgxQixg8l4R8YQAoKcxHdakp1E8GFT3D40m530wx955r4G4AZwK4BMB6ADUAhmAKuT8H8E1V7Ym7tUVVnxORCwH8O4CXwRSdj8PkOn2W+YdElC3ae0ewbV9n6PwbNzJGgogoG0XkCy+f7+v4hGs3LcaPHjoEALj7mQ50DYyhvjI5b7oesLqFNy2fh7KiZLwN8d76JTW4xclk9iJneF/HAIad3OaF1SVYUJ2c7ikyRORDACpgBqFfCPOedw+AL0bdbinMe9ZfqOpv5/B48eInVs/2Pokykd0x3J6igajpFpEvXJv8wnBkxnD2dgxH5AtXJm+f11BZHIot6egfRV1Fcl7/ZII5vyJT1ScBvHsGtz8MIO4rYlVtBfC2ua6LiMjPbtlxNDR07pIz67AswwfwEBFRbJmQL+w6s6ESG5fVYufhU5gMKm5/rA3/vOWM6TdMwLaofOFsYUc3tBzthaqmtfhvdymzWzglPgSg0Tr/JwDXq2roH7QzH+f/YIbNvTe9yyPKTk014aNr2k4Ne7iS5InoGE5BYdgukmZ1x3BElETyireNVSU40GlmhnYOjGFN0u7Z/3jcMhFRmk0EgriZQ+eIiLJeIKjYHpEvPM/D1STmWusIllt2Hk3KNPiJQBAPHegOnc+GfGHXmfUVqHQG33QPjqX9kOcWe/Dckpq0PnYuUNUFqioAFgB4A4AVAFpEZIN1sw8AuBzAO+Z6hKuqXhDrC8DeudwvUaaJ6BjOkiiJ9t5wgbu5NvmxUnaRNJuHz9kdw8nKGAaicoaz+PmLhYVhIqI0u+eZTnQOmB1aXUUxXnxO4zRbEBFRJnrmeD8GRk2+cENlMZZnwNEhrzhvISpLTKHz8Mlh/O3g3KNSW472hnKWF1WX4ExnYFs2yMsTnG8VZNMdJ9FidQyvZ8dwyqhqh6r+GsBLAMwH8DPADGIH8HkAP1HVOz1cIlFWYcbwzEUMnxvI4o7hiIzh5BWGG3MkozkWFoaJiNLMHjr3dxc2ozCff4qJiLJRRL7wCn/nC7tKi/LxemsI3c1JGEJ3//5wpv7lZzVkxPMwE3ZBtiWNA+h6hsZxqHsIgBnod25TVdoeO1ep6hEATwNYIyJ1AM4BUAzgbSKi9hdMFzEAHHAue503qybKPMwYnrn55UXIc3avPUPjGJ8MJv0x/MAuejckaQ4CADRW5kbHdSzZMfWBiChDtPYM48EDJpZOBHjjJsZIEBFlq0cOZlaMhOvajUvws78dAQD86ckTODU0jtryolnf3/37szNf2LXBo47hx1vDj3XOoioUF+Sn7bFz3CLnNADgMIAfx7ndK2EiKG4D0O/clogSUFdejKKCPIxPBtE3MoGB0QlUlhR6vaxZU9WIjuFUFIYL8vNQV1EcKpx2DY6lpDPZa3bRNrlRErnRcR0LC8NERGl0046jcOMaL11Zj8Xzkp8vRURE3gsEFTsOZc7gOds5i6qwrrkau9v6MB4I4lct7fjHS5bP6r46B0bxZHs/AKAgT/D8MzPneUjU+sXhjuGnj/VjdCKAksLUF2l3HekNfc/Bc8njxEN0qGpf1OV5AD4HoAHAw06e8CkAb49zP9tgCsOfUNVnU7pooiyTlydorinFQeeoiPbeEaxekLmF4VPDExiZCAAAKooLUF2amp+loSpcGO7oH826wrCqRmQMJ7VjmBnDRESUahOBIG59tC10/jp2CxMRZa29J/rR7+QL11cWY0UG5Avbrt2UnCF0D+4PD527YGltRnd8xVNdVogz6s3vdzKoeKK9b5otksPuTt6wtCYtj5kjXgHghIjcJSI/EJEviMj/AjgA4BMATgB4h6crJMoBETnDPZkdJ9F2Kjx4rqmmNGWRSo2VVtdrFhY3e4cnMB4wERmVxQUoL05er2tDJTOGiYgoxe56ugPdg2Yn01hVjCvObvB4RURElCp2jMTm5fMyLlf31esWoazIdL3u7xjErllm526zYyTOyr4YCZfdsbvrSOrjJAJBxe7W3tB5Dp5Lqrth4iHqAbwBwIcBXAmgB8BnAaxR1ae9Wx5RbsimnOFU5wu7GrJ8gFrHQLjYXV+VvG5hwHRbu7oGxxAIzu4D8UzEwjARUZrcuD08dO6aCxdz6BwRURazB89lUoyEq6K4AK9euyh0/mZrcGqiAkEN5eoDwJZV2fuBaLoH0O3vGMDQuDksubGqGIuqk5ezmOtU9UlVfbeqnq+qdapaoKrVqrpRVbeqas/09wKo6hZVFcZIEM2OHYOQ8YVha/1NKSwMR8QhDGRfx7AdI2F3RydDcUE+asrMUU2BoKJnaDyp9+9nrEoQEaXB4e4hPPSsOZw2T4BrGCNBRJS1gkHFjkP24LnMKwwDwLWbFoe+//2e4xgYnZjR9rvbetE7bLZpqCzG2Qsrk7o+P7GjHHYdPTXr6I1ERcRILKnNuI50IqLpNNeGZ7HYUQyZKNWD51zZHocQOXguuR3DQGSxuSMLozjiYWGYiCgNbtoZ7rTaclZD1g0CICKisL0nBtA3YgqidRXFofzZTHP+4hqsXmCKuSMTAfz28WMz2v7+fVaMxKr6rC5ermyoRIWTddg5MJby7ja7K3n9kpqUPhYRkRfszlo7iiET2YXhpprUDR+3i6XZWNh0B+sBQGNV8o+Uacjyjut4WBgmIkqx8ckgbufQOSKinGHHSGxekXn5wi4RwbUbw13DN++cWZxEruQLA0B+nmDd4urQ+VTHSUR3DBMRZRu7s7Yt4wvD4Y7nVHYM28XSzizvGK6vTEHHcJZnNMfDwjARUYr9+akTOOlkFC2sLsGWLH9zTESU6zI9X9j2+vXNKC4wbxmebO/Hk+19CW3XMzSOPW29AEyE0qVnZv++L2IA3dHUDaDrHR7Hwa4hAEBhvuDcpupptiAiyjwNlSUoyDMfrJ4cGseIk6ueidKVMZztHa8RGcOp6Bi2is3ZWFiPh4VhIqIUixg6t3ExCjh0jogoawWDih2Hw/nCz1sxz8PVzF11WSFecd7C0PmbEhxC9+CBLrgxu+uX1KLaGeiSzSILw70pe5yW1vB9n7OwCiWF+Sl7LCIir+TnCRbWhIt/7b2ZmTPcNzKBgdFJAEBJYR7mlxel7LHmlxfDqaXj1PAExiYzt5geS8eAnTGc/MJwRMdwFhbW42F1gogohQ52DeJvTudYnpjCMBERZa99HQOhgWt1FUU4o77C4xXNnR0n8bvHj2F4fHLabex84S2rsr9bGDCZzK6nj/VhdCI1b8hbjoS7kdczRoKIslhzjT2ALjPjJNoj8oVLUxovlZ8nEREL2db1GtkxnIooCfu5Y2GYiIiSwO6seuHqRiys5tA5IqJsFpEvvHx+xuYL2zYtn4cVdWaA3sDYJP6w5/iUtw8GFffnUL6wq7a8KPQ8TQQUTx1LLHZjpuxu5A1LWRgmouzVlAU5w5H5wqkbPOeKyBnOoq5XVY34eRoqUzF8jhnDRESURKMTAdz+WHjo3Js2c+gcEVG2i8wXzuwYCZeIRBzxcvPO1ilv/9Sx/lC2/vzyIpy7KHcycO0O3l1HepN+/4Gg4nErSmK91aVMRJRt7EFtdk5vJklXvrDLLphmU8fwqeEJTARMRlVlSQFKi5Ifo2RnDHewY5iIiObqz0+dwCnncOKmmlJcliOH0hIR5apgULH9UDhfONMHz9muvKAZhfmm+/mxI6ewv2Mg7m237esMfX/Zqnrk5WV+13Si1i+pCX3f0pr8AXTPdg5icMxEedRXFqd0uj0RkdeaarKhYzi87nT8zbYH0GVTcdP+WVKRLwwgIoaje3AMgaCm5HH8hoVhIqIUucEaOnftxsXIz6E3xkREuWh/ZzhfeH55Ec5syPx8YVddRTFefE5j6PzNO+J3DdsxEltyJEbCtSHFHcO7joaLzRuW1GRFVAkRUTx29EL7qcwcPhedMZxqjZX2ALXs6RiOLAwnP18YAIoL8jHPGQ4YVODkYPY8f1NhYZiIKAWe7RzADqdrLD9P8HccOkdElPUeec7KF14xL+uKdtduDEci/aqlLeZwtb7hiVDxUgS4dGVuFYbPWlCJMufw1hP9oziW5EOfW45y8BwR5Y7mbMgY7k13xnB2dgxHDJ5LQb6wKzJOgoVhIiKapRu3hzupXnR2Q8oOdyEiIv945GB2xki4LjmzLvQmvXd4An9+6sRpt3no2W64R16uba4Jdd7kivw8wbrmmtD5FmtQXDJEDJ5jYZiIstyC6hK4B112DoxhbPL0DyT9rj3NURL2+86uLOoYtgfP1aeoYxiIfP6yqbA+FRaGiYiSbHQigF/uCg+du27zUg9XQ0RE6WDyhe3Bc9lXGM7LE1xzoTWELkacxP37w/nCl+dotv6GpTWh7+3oh7nqG57As52DAICCPMF5Tbkz1I+IclNhfh4WWIW6Y72ZVagbGpsMzZwpys9DfUXqCpqu7M0YTn/HcGcWFdanwsIwEVGS3fnEcfSNmBcAi+eV4tIz6zxeERERpdqBzsHQm7955UVYmUX5wrarL1wc6t7628GTONw9FLpOVSPyhXO2MGznDCexMPx4W2/o+7MXVqVkIjsRkd9E5gxnVpxEuxUntKimJC3DWBvsjOEsikJIx/C56PvOpsL6VFgYJiJKssihc0tyaho7EVGueuSglS+8PPvyhV0LqkvwwtUNofO3PBruGt57YiD0JrS6tBDnL65J9/J8wf65n2rvT9qhz7uORA6eIyLKBU0ROcOZNYDOXm9TGmIkADP81h163jcyEXMeQCayB+mlavhc9H3b8RXZjIVhIqIk2ndiAI85b9wK8gR/dyGHzhER5QK7MJyNMRI2ewjdbY+2YSIQBABs2xfuFr50ZV3ojWmumV9RjGXzTYfbeCCIp471J+V+7e7jDUuZL0xEucHO5W1P8kDPVIvIF65J/eA5wMQ+2XEI2ZIz3GV17zakMkqiKjs7rqfCwjARURLduP1I6PuXrlmA+srU50gREZG3VBXbD2X34DnblrPqQx013YNjuOcZkyts5wtvOash5ra5IiJO4sjc4ySCQcXjrb2h8+sXszBMRLmhqcbuGM6swrC93nR1DAPRxc3M73oNBjUi77chhR3DkRnDmf/cJYKFYSKiJBkZD+BXLe2h89dtXjLFrYmIKFsc6BxEz9A4gOzOF3YV5Ofh6gusIXQ7j2JgdAKPHg4XQC9bldv5+uutqIcWq6A7W891DWJgdBIAUFdRhMXz0ldgICLyUiZnDLdZHc7N6SwMV9oD6DK/67VneByTQQVgoqpKClOXsd/IjmEiIpqtO/YcC71pWza/DM/L8o4xIiIy7BiJTcvm5US2/DUbw4Xh+/d34fbH2kJv2tYsqkrpYZ6ZYL3VMdyShI5hO0Zi/ZLarM2wJiKKltkZw1bHcE36CsN2Tm42dAxHDp5L7RG59hG/3YNjmHTisrIZC8NERElyozV07o2bOHSOiChXROYLz/NwJemzeF4ZLl1puoJVgS//aV/oustX1Xu1LN9YvaASpU5H07G+UZzom9sb85ajvaHv13PwHBHlkEU14Q8aT/SPhnLtM0FExvC89GQMA0Cj9eFsRxbEIXRanbup/uC5MD8PdRVFAMzrm+7B8ZQ+nh+wMExElARPH+sPZf8V5efhqguavV0QERGlhapi+0ErX/iM3DlaxB5CN2JNPc/1fGHAxG2sba4OnW85Oreu4YjBc0uYL0xEuaO4ID8UjRBUzPmDtnQZnQige9AUNPPzBI1pnD1jxyF0ZUEcgp31m8p8YVe9VXzOhZxhFoaJiJLgxh3W0LlzF2B+BYfOUWqIyEoR+aiI3CsirSIyLiIdIvJbEXlBnG2uFxGd4utd6f45KPfct7cT/3nnMxnzhi5Rz3YO4qSTL1xbVohVDZUeryh9XnxOI+aXF0VcVllcwI5Wx4al1gC6ORSG+0cncKBzEIApLtgFZyKiXNBcm3kD6NqtfOGF1SUoyE9f+a3ejpLIgsKmnfVrF71TJTKKI/ML69Mp8HoBRESZbmhsEr9pORY6f90mDp2jlPocgGsAPA3gTgA9AM4C8BoArxGR96nqN+Ns+1sAj8e4/NEUrJMo5Olj/fjH/9uJoAJ3Pd2BP77v0pQODkmniHzh5bmRL+wqKsjDlRc04wcPHAxddsnKOhSm8c2vn61fXBP6fpcVBTFTu1t7oSa+GasXVKKsiG/hiCi3NNWWhf6Ompxh/x+d41W+MBAVJZEFhc2IjOE0dF5HPn+ZX1ifDl9VEBHN0R27j2FwzAydW1FfnjP5kuSZPwH4kqq22BeKyOUA7gLwFRG5TVWPx9j2N6r60zSskSjCN+85AGcuGQ51D+G//rIPn3zlOd4uKkkesWMkcnDo6DUbF0cUhpkvHGZ3DD/R3ofxySCKCmZeNN91pDd8n4yRIKIcZHcM2524fhaRL1ybvnxhIBuHz1kZw2nuGO7MgudvOvw4n4hojm7cER46d92mJZwUTimlqj+NLgo7l98PYBuAIgAXp3tdRPE8fawff3rqRMRlP37oEB47MrfMVT9QVWw/ZA+ey73C8Bn1FbjMKQaXFubjhauZL+yqqyjGEmfY0PhkEE8f75/V/UTkCy+tScbSiIgyit1xmylREqaz2WiqTW/HcG1ZEQrzzXvSgdFJjIwHptnC37qsOIzGdGQMV2VXx/V0WBgmIpqDJ9v7sKetD4BzSO0GDp0jT004p5Nxrj9fRN4vIh8TkbeICP/BUsp9854Doe/dlIWgAh++fTdGJzL7jcpzXYOhadU1ZYU4qzF38oVtX/u7dfj4y1fjF2/fnJZOnkxi5y3vmsWHIcGghobbAsD6xewYJqLcE9ExnCGFYbuzuTnNheG8PEG9NfMm0weoRXQMV6ahY7gye567RMy5MCwi80Xk7SLyaxF5VkRGRKRPRB4SkX8UkYQfQ0QOTzEY58T090BElF43bA93C7/i3AWojRrCQ5QuIrIUwBUAhgE8EOdm7wPwNQBfAPAzAIdF5HsikvArLBF5LNYXgNVz/BEoC0V3C//gLReivMhkCx/sGsLX7t7v1dKS4m9WjMSmZbmVL2ybX1GMd15+Bi5YyqJlNDv6ocUq8CbqYPcQ+kbMZ37zyouwdH56D0cmIvKDiOFzvcNT3NI/7M7m5jRnDAORkQuZ3PUaCCq6Bu0oiTRkDGfJc5eoZGQMXw3guwCOA7gPwFEAjQDeAOBHAF4uIleruiMTptUH4OsxLh+c+1KJiJJncGwSv3u8PXT+us1LPVwN5TIRKQZwA4BiAB9R1ei2tEMA3gPgLwDaAFQDuASmQPxOAFUArkvbgiln2N3CL1uzAC86pxGfeOXZ+OSvnwQA/PCBg3jZmgVYn6G5qfbguVyMkaDp2YXh2XQMR8RILKlhXBUR5aSmmvCHYsd7RxEIKvJ9/mGslxnDQPbkDJ8cGkPAGVRRW1aI4oLUDy+2C8O50DGcjMLwfphJ6H9Q1aB7oYh8AsAOAFfCFIl/meD99arq1iSsi4gopX77eDuGnLymMxsqsHFZZhY2KP1E5DCAmXyScIOqvjnOfeUD+DmA5wO4BcBXo2/j5A/fb100DOA2EXkEwG4AbxSRL6nq7ukWoqoXxFnHYwA2TLc95Y7obuH3XrESgMli/8Oe43j4uZNOpMQe/P49l6CkMPUv9JNJVbGdhWGaxuqFlSgpzMPoRBDtvSPo7B+dUdxGy9He0PeZ+gEKEdFclRblY355EU4OjWMyqOjoH8UiD7pwEzU+GUSHU1AUARZUpz9mKbK4mbldr51pjpEAgLqKIogAqkD34DgmAkEU5mdvEu+cfzJVvVdV77CLws7lJwB8zzm7Za6PQ0TkJ6qKG7dz6BzN2nMA9s3g61isO3GKwr+AOXrnVgBvnsEROlDVVgB3Omcvm80PQhRPdLfwOYuqAAAigi9duRZlTqTEs52DEbfNFHa+cHVpIVYvyM18YZpaYX4e1jbVhM7vsgq9iWixOobtvGIiolwTkTPc6++c4eN9I3BfkS+oKkFRQfqLihGF4QzuGLY7dtMRIwEABfl5mF8efqzuwcwtrCci1f86pxuCE0uxiLxZRD4hIu8TkRc4b3yJiHxjT1sfnjpmposXc+gczZCqXqGqq2fw9ZHo+xCRQgA3AbgWwI0ArlPVmexvXV3OafnsfyKiSPG6hV2L55Xh4y8Px1J/7/7nsHsW+ateesTOF16eu/nCNL31S2tC39uF3ukMjE5gX8cAADO4cV1zzdQbEBFlsSY7Z/iUv3OG7XzhJo86m+srsyNKws74bUzjgNvIKI7sLgwnI0oiJhEpAPD3ztk/zWDTBTCHxNoOicjbnENhE3nsx+JcxcE4RJQUdrfwK9cuRHVZoYeroVwjIkUwHcKvhRki97boI3dmYLNzejAZayMC4ncL2960eSn+8MRxPHKwx4mU2I073nNJWrLjkoH5wpSo9YutnOEZFIb3tPWFOs7OWlCF8uKUvXUjIvI9O6fXzu/1o8h8YW8Kw9kyQM0uajemqWPYPFZJqBEskwvriUhlx/AXAZwL4E5V/XOC2/wEZqL6ApjOpfMAfB/AMgB/FJF1KVgnEdGM9I9O4He7w0f2v4lD5yiNnEFzv4YpCv8YCRSFReTCGJflicjHATwPQDdm9iEuUVzTdQu78vIEX75yHUqdbOH9HYP41j3PpmWNc6WqER3DF62Y5+FqyO82WB3De9r6MBFI7HM8e1jdBsZIEFGOsztv23xeGLY7mps8KwyHi6iZPEDNzkdOV8YwEPX8ZXlhOCUfO4vIewF8EMBeAG9JdDtV/WzURU8CeJeIDDr3txXA6xO4Hw7GIaKU+W1LO0YmzNC51Qsq+WaN0u17AF4BU8xtB/DpGPnW21R1m3V+p4g8CTNorh1ANcywunNhBtG9SVX7U7xuyhGJdAu7lswvw8devhqf+d1TAIDv3v8cXrpmAc5rrk75Oufiua6hUN5cdWkhzl4Q/2ckaqgsQXNtKdpOjWBsMohnjvdjbQKxEHZ38QYOniOiHJdJGcNtvXbHcNkUt0ydxko7YzhzO4Y7PeoYrq/MjuF9iUh6YVhE3g3gGwCeBnCFqvZMs0kivgdTGOZgHCLylKriBnvo3GYOnaO0W+6c1gH49BS322Z9/1UAmwC8EMA8AEEARwH8D4D/VlXGSFBSJNotbHvLRSZSYsehHgSCig/fvhu/e/clngxqSdT2Q+EYiY3LmC9M01u/pDbU4bbryKlpC8OqihYrd5uD54go10VmDPu8MOyDjOGaskIU5edhPBDEwNgkhsYmMzKSyI7BaPAsYzi7O4aT+opbRN4P4Fswnb4vUNUTU2+RMA7GISJfaGntxd4TZhBMaWE+Xre+yeMVUa5R1S2qKtN8bY3a5sOqermqLlLVElUtc4bavZtFYUqmmXQLu0ykxFqUFJqXpXtPDODb9/k7UoIxEjRT9tFFLQkMWjzUPYTeYTPHu7asEMvr+DaIiHKbXWBt7x1BMKgermZqfsgYFpGIAXSZ2vUamTGcxsJwZXZkNCciaYVhEfkogK8BeBymKNyZrPsGcJFzyjevROSpGx4Jdwu/et1CVJVw6BwRETC7bmHXsrpyfOSl4RnB37nvWTzZ3pfU9SWLyRfm4DmaGTsKIpEBdLuO9oa+X7+klkcnEVHOqywpRHWpee81PhkMRTr5zWQgiBNWMXORRx3DQObn5AaCGvF7rq9I7/A5FzuGEyAin4IZNvcYTHxE9xS3LRSR1SJyRtTlZ4vIaR+Fi8gyAN92zv4iGeslIpqNvuEJ/H5PeOjcdRw6R0QUMptuYdv1Fy/DxmWmeDYZVHz49j0Yn0xsSFc6HeweQpfTdVNVUoCzFzJfmKZ39sIqFDvxKK09I6F/Q/G0WMXj9YtrUrk0IqKMYXfftvk0Z/hE/ygCTjdzfWUxSpwhu16IKG5mYMfwycExuI3h88qL0hoz1mAV1afbZ2e6OT+rIvJWAP8OIADgQQDvFZGtUV/XW5s0AXgGwD1Rd3UNgBMi8gcR+Y6IfElEbndueyaAO2EyEomIPPGrljaMOUWKcxZWYZ3PhyMREaXLXLqFXXl5gi9ftS5UPHvmeD++u+25pK0xWbZbMRKbls9DPvOFKQFFBXk4ryn8uqFlmq5hu2N4w1IOniMiAiLjJPyaM+yHfGFXQ2VmdwxH5AtXpq9bGADmlxfBfYl3cmjcl80KyZKM5Gl3CE4+gPfHuc39AH46zf3cB+AsAOthJqWXA+gF8BCAnwP4uar6N0SGiLKaquJGDp0jIopprt3CruV15fjwS8/Cf/zhGQDAt+49gBef0zjr+0sFxkjQbG1YWotHj5iC8K6jvXjJmgUxbzc4Nol9J/oBACLAOnYMExEBAJpry0Lft/u0MOyHfGFXQ4bHIXiVLwwABfl5qKsoDmUzdw2OeV7oT5U5dwyr6tYEhuBssW5/2LlsWdT93K+qb3SG4dSoaqGq1qvqi1X1ZywKE5GXHj1yCgc6BwEAZUX5eO35izxeERGRPySjW9j2tucvDw3qMpESuzER8EeXBvOFaS7sSIipcob3tPWGDp09q7ESFRk4RZ6IKBWaau0BdMMeriS+diviosnjwnBkTm7mxSF0DIQLw+nuGAZyJ2c4fQEdREQZzO4Wfu35i1DJoXNERACS1y3sys8TfOXqdaEcuaeO9eN7PomUONQ9FOocqWS+MM2QHQmxp60Xk3E+8GiJGjxHRERGRMawTzuG206FC9Z2h7MXIobPDWReYbPTKmanu2MYiI7iyLzCeqJYGCYimsapoXH84YnjofPXbeLQOSIiIPndwq4z6ivwoZesCp3/5r0HsNc5tN5L2w9Z+cLLmC9MM9NYVRI6DHV0Ioi9JwZi3m7XkXA3sds9T0REkZm9vo2SsDqGmz3PGA4XUzOxsGkXs+0id7rYURyZWFhPFAvDRETT+OWutlDY/HlN1TiPQ+eIiAAkv1vY9o+XrMB6pyg2EVB8+LY9cTss04UxEjRX51uF3lhxEqqKltbe0Hl2DBMRhS22OnDbTo3Aj4mjbT7KGLaLqZkYhRAxfM6DjuFMf/4SxcIwEdEUVBU37ogcOkdERKnrFnbl5wm+ctXaUKTEE+19+P4DB5P6GDPBfGFKhg1WodeOjHAdOTmMnqFxAEB1aSFW1JWna2lERL5XVVoQyl0fmQjg1PCExyuKFAwqjvkoY7i6tDD0OmpoPIDBsUlP1zNTdjHW+4zhzOu4ThQLw0REU9h+qAcHu4YAABXFBXjNOg6dIyICUtst7DqzoRIfeFE4UuIbdx/A/o7Yh9+n2uGTw6E3BZXFBSn5eSn7bZimY9i+bP2SGuQxroSIKEREonKG/TWArnNgDBMB08U8r7wIZUXeDg8Vkcic4QzrenXnOgDeZwyzY5iIKEdFD50r52RwIqKUdwvb3nHpcqxzInzGA0F8+LbdnkRKbLe6hTcuZ74wzc45i6pQlG/egh05OYzuwcgOpIjBc4sZI0FEFM3POcPtveFCdZPH+cKuxsrM7HqdDAQj9pH1HncMdw1kznM3UywMExHF0TM0jj89GS58MEaCiMhIR7ewqyA/D1+5el2omLa7rQ8/fPBQyh4vnsgYiXlpf3zKDsUF+Ti3Kfz/5fGoOAm7Y3jD0po0rYqIKHNEdgz7qzDsp3xhV4PdMZxBA9S6B8fhRkjXVRShMD/95csGZgwTEeW22x9rxbjTlXb+4hqsWcShc0RE6ewWdq1qrMT7XhR+nK/dvR/PdqYvUsLkC/eEzjNfmObCzhm2C8HD45PYe8L8uxYB1i2uSffSiIh8z87tbe/1b2HYLx3DDREdw5lT3IzMF05/jAQAzC8vDh0hdmp4AmOTAU/WkWosDBMRxaCquGlHa+g8u4WJiIx0dgvb3nnZCpzX5ERKTAbxodv2IBBMzzTyIyeHccJ5g1JZXIBzFjJfmGZvfZwBdHva+kL/plc2VKCqpDDdSyMi8r2mmrLQ937LGPZjx7Adh9CZQVESdr6w3bmbTvl5gvoKO6M5c56/mWBhmIgohr89dxKHus3QucqSArx6LYfOERF50S3sMpESa1GYbzo3Hm/txY8fOpiWx95+KBwjceGyWhR4cDgjZQ87ImJ3W28oMzsiRmIJ84WJiGLxc5SE3cHcVFs2xS3Txx4+15FBObl2x3CjRx3DQHQUR+Y8fzPBV7VERDHcsCM8dO4N65tQWpTv4WqIiPzBq25h1+oFVXjvC8PF6K/+ZT+e6xpM+eMyRoKSaWF1KRZWmze5w+MB7Osw8RG7jvSGbrN+SY0HKyMi8r+IKAmfFYbtDma/dAxnapREp10Y9qhjGIh8/joz6PmbCRaGiYiidA+O4S9P2UPnlnq4GiIif/CyW9j2ri1nYI1TkB6fDOIjt6c2UsLkC9uD51gYprmzC78tR3uhqni8lR3DRETTmV9ehJJCU8oaGJtE38iExysyVDWiUN3kk8KwXVTNpMJmR78dJeFdx3BjDgygY2GYiCjKbY+2YSJgigwXLK3FWQsqPV4REZH3vO4WdhXm5+ErV61DgTMM5LEjp/CTvx5K2eMd7RnG8T7zRqCiuCBUlCaai+gBdK09I+geHAdgIqzOqK/wamlERL4mIhGD3fySM9w9OI6xSRMNVFVS4JuceLuo2jkwBtX0zGeYq84Be/icdx3DdkZzJkVxzAQLw0RElmBQcZMVI3HdJg6dIyLyS7ew65xFVXj3C88Mnf/Kn/eFcuGTbbsVI8F8YUqW6I5hO1/4/MU1yHM++CAiotM1W/m9fomT8GO+MGCK1G6H9fB4AINjkx6vKDF2x3Cjhx3DdlGaw+eIiHLAX5/rxtEe86lzdWkhXrl2occrIiLynl+6hW3/suVMnL3QrGNsMogP37Y7JZESjJGgVFizqDo0SPFQ9xDu3dsZuo4xEkREU2vy4QA6u3PZ7mj2mohE5QxnRnHT7hj2sjDcGNFxzSgJIqKsd+N2a+jchiaUFHLoHBHlNr91C7uKCvLw1avXhiIlHj1yCv/38OGkPgbzhSlVSgrzsWZRdej8nU8cD32/YSkLw0REU7EHu9mdul6yO5f9MnjOlWk5wxOBYCheSQSoqyjybC0NzBgmIsodnf2juOvpjtD5N21mjAQRkR+7hV1rFlXjX14QjpT48p/34nASIyVae0ZwzMkXLi/Kx7k++tkp89lxEpNWt/v5zTWn35jSSkS+JCL3iEiriIyISI+ItIjIZ0RkftRtV4rIR0XkXuf24yLSISK/FZEXePUzEGUzP2YMt/m4MNwQkZPr/+Jml5XlO7+82NMYr4iM4Qzptp4pFoaJiBy3PtoaemO2adk8nNnAoXNElNv82i1se/cLzsRqZ0jo6EQQH/nlHgSTFCnxyKFwt/CFy+YxX5iSKlZkxJkNFagu88fAohz3AQDlAO4C8A0ANwCYBLAVwB4RWWzd9nMAvgigEcCdAP4LwF8BvBLAvSLy3vQtmyg3RGQM+6VjuNe/heFGK0oiE3JyOwfsfGHvBs8BwLyyotDRaX0jExidCHi6nlTgq1siIgCBoOKmHa2h89exW5iIyNfdwq6igjx85ap1yHdetO841IOfP3IkKffNGAlKpViRERusLmLyVJWqXqSq/6CqH1PV96jqRgD/CWARgI9bt/0TgA2qukZV36mqH1fVNwC4AsAEgK+ICIdWECVRs+8zhv0zfA6IjkPwf2HYjmzwMl8YAPLyBPXWADq7mzlbsDBMRATggQNdoU95a8sK8bJzF3i8IiIib2VCt7DrvOZq/PPlZ4TOf/GPe3H05NwOLVVVbD/YEzp/0Yp5c7o/omiLqksipp0DwHoOnvMFVY13rPWtzulK67Y/VdWWGPdxP4BtAIoAXJzsNRLlsvqKYhQ5R/H0Dk9gcGzS0/WoasZkDGdClERnRGHY245hICqKIwtzhlkYJiJC5NC5Kzc0c+gcEeW8TOgWtr3nijOxqrECADAyEcBHfrl7TpESbadGQh8Ylhfl49ym6mm2IJoZETktTiJWvAT5yqud0z0J3n7COfW2akWUZfLyBItqwsW6do+7hnuHJzA0biIGyoryUeOzSKDIKAn/Fzbtrub6Sm87hgGgsTKzOq5nioVhIsp5J/pGce/eztD5NzJGgohyXCZ1C7uKC/Lx1avDkRKPHOzBDTuOTrNVfHaMxAXL5qGQ+cKUAhuW1oS+rywuwMqGCu8WQ6cRkQ+JyFYR+ZqIPAiTJ7wHJlN4um2XwsRJDAN4IMHHeyzWF4DVc/gxiLJSZM6wtwPoovOFRcTD1ZzO7njtzIAohM4Bv3UM24Vh/xfWZ4qvcIko592ysxUBp6vsohXzcEY935QRUW7LtG5h19rmGvzTZStC579w5zNo7Zndm8VHGCNBaXDxGXWh7zevmIe8PH8VEwgfAvAZAO8HcAlMnvBLVLVrqo1EpBhmYF0xgK2qeirF6yTKOU01/skZjswX9leMBBAVJdE/CtXkDOlNFbsrt9EXHcOZVVifKRaGiSinBYKKW3aGO8qu27zUw9UQEXkvE7uFbe+7YiXOdLouh8cD+Ogv98zqDRAHz1E6nNtUjc++Zg3esL4J//bKc7xeDkVR1QWqKgAWAHgDgBUAWkRkQ7xtRCQfwM8BPB/ALQC+OoPHuyDWF4C9c/pBiLKQnePrdZREW0S+sL8GzwFARXEBSp2oxNGJIPpH/Z1u46fhc9FryIQojpliYZiIctq2fZ041mf+uM8vL8JL1zR6vCIiIm/Z3cIvXdOYMd3CrpLCfHzlqrVwGy8ffu4kbpxhpERrz3DosNCyonycx3xhSqG3XrwM/33N+VhWV+71UigOVe1Q1V8DeAmA+QB+Fut2TlH4FwCuhhlU92b1e2seUYZqqvVTx3D48Zt8NngOMJn2dtew34ubdleu76IkMmB430yxMExEOc0eOnfVhc0oLuDQOSLKXZneLexav6QW77AiJf7zD89EHOY5ne2HwjESFyytZb4wEQEAVPUIgKcBrBGROvs6ESkEcBOAawHcCOA6VfV3Wx5RBrM7c9t6vS0MR2cM+1Gm5AyPTwbRMzQOAMgTYH6F94Vhu2OYw+eIiLLIsd4R3LfPGjq3kUPniCi3RXcLr1mUuZ2yH3jRKpxRbzowh8YD+Pivnkg4UoIxEkQ0hUXOacC9QESKANwG0yn8MwBvUdVAjG2JKEmaIqIkvB0+F9Ex7MOMYSC6uOnfrteuwXDhta6iODRU2EsNlZnTbT0bLAwTUc66eWcrnJlzuOTMOh7CSUQ5LVu6hV0lhfn48lXr4A4Gf/BAN27Z2ZrQtiwME+UuEVklIqd9KiYieSLyeQANAB52B8o5g+Z+DeC1AH4M4G2qGkznmolyUWNlMQqcomH34DhGJ7z7LMYuTPsxYxiILG76uevVb/nCAFBbVoTCfPNvrX90EiPj2fW5X4HXCyAi8sJkIBg1dI7dwkSU27KpW9h1wdJavP2S5fjhg4cAAP/xh2dw6ar6Kbt52k4Nhzp/SgvzsbY5858HIpqRVwD4gog8BOAQgJMAGgFcDjN87gSAd1i3/56zTTeAdgCfFjmtw22bqm5L7bKJcktBfh4WVJeE9tltp0ZCw2fTqX90IjTMrbggD3UVRWlfQyLsrF4/dwx3RhSGvY+RAIC8PEFDZUkoMqRzYBRL52dPUxkLw0SUk+7d2xn6pLSuohgvPodD54god2Vbt7Dtgy85C/c804mD3UMYHJvEx3/1BP7vbRsRo3ADANh+MJwvfOEy5gsT5aC7AZwJ4BIA6wHUABgCsB/AzwF8U1V7rNsvd07rAHx6ivvdluyFEuW65trSUGG4vdebwnB71OC5eK8vvGZ333b5OGPYzj+ur/RHxzBgBtC5heGO/rGsKgzzlS4R5SR7Qv3fXdjMN/6UMURkmYjoFF83T7HtW0Vkh4gMikifiGwTkVelc/3kT9nYLewykRJrQ5ESD+zvwm2PtcW9PWMkiHKbqj6pqu9W1fNVtU5VC1S1WlU3qurWqKIwVHWLqso0X1s9+nGIslpTjTWAzqOc4UzIFwaAhsrMyBju8GHHMBCVMzzg3+dvNtgxTEQ5p7VnGPfv7wIAiABv3MQYCcpIuwH8JsblT8a6sYh8FcAHAbQB+CGAIpjJ6XeIyHtU9dspWif5XDZ3C7suXDYP//D85fjxQyZS4nO/fxqXrqzDwurT38A9csguDM9L2xqJiIhoZpojBtCNTHHL1MmEfGHAdLy6Onxc2LTzj/2SMQxED+/zb8f1bLAwTEQ555adrXAH01+6sh6L5/l3B040hccT7UASkYthisLPAdhoDcz5CoDHAHxVRH6vqodTtFbysWzuFrZ96CVn4Z5nOnD45DAGRifxiV89gf+9PjJSou3UMFp7wvnC5zXVeLRaIiIimk6TVRhu86gwbD+uXaj2m+jCpqr6MvbCrx3D9vPX6eOO69ngsdNElFMmAkHc8mh4Kv117Bam3PAu5/TzblEYAJxC8P8AKAbwNg/WRR7LhW5hV2lRPr581bpQpMR9+7rwy13tEbex84UvWFqLogK+VCYiIvKriI7hXo86hnszozBcUVyA8qJ8AMD4ZBD9I5Meryg2O/+4wU8Zw5WZMbxvNub8aldE5ovI20Xk1yLyrIiMOLmFD4nIP4rIjB5DRJpF5H9F5JiIjInIYRH5uojUznWtRET3PNMR2tk0VBbjirMbPF4R0awtEpF3isgnnNO1U9z2hc7pn2Jc98eo21AOyZVuYdem5fPw1uctC53/9zueinhxv50xEkRERBmjmRnDMxLRNezTOAn7dVmDjzqGG+yOYR8P75uNZERJXA3guwCOA7gPwFEAjQDeAOBHAF4uIlerugduxyciZwB4GEADgN8C2AtgE4D3AXiZiDxfVU9OcRdERFO6YXt46Nw1Gxdz6Bxlshc7XyEisg3AW1X1qHVZOYAmAIOqejzG/biVwVWJPKiIPBbnqtWJbE/+kUvdwraPvOws3Lu3E0d7htHvREr86K0XQkTwiNUxzMFzRERE/ragugR5AgTVFOvGJgMoLshP6xoiO4b9HVFYX1mMg91DAEwBdlVjpccrijQ2GcCp4QkAQH6eYH65fwrDdqwFO4ZPtx/AawA0q+qbVPXjqvoPMG8QWwFcCVMkTsR3YIrC71XV16nqx1T1hQC+BuAsAJ9PwnqJKEcdPTmMBw90AzBD567ZuNjjFRHNyjCAzwG4AECt83U5zIezWwDc4xSDXW4LaF+c+3Mvr0n2Qsnfcq1b2FVWVIAvXRlusL9nbyd+83g72ntHcLTHdBuVFOZhbXONRyskIiKiRBQV5IW6YFWB473pLdgNj0+iZ2gcAFCYLxFxA37k9wFqndaa6iuKkZ/nnwzkxko7Y9h/z91czLkwrKr3quodqhqMuvwEgO85Z7dMdz9Ot/BLAByGyTu0fQbAEIC3RL3ZJSJK2E07w93CW1bV+/4TXcpeTkySzuDrF+62qtqpqp9W1V2q2ut8PQCzD90O4EwAb0/V2lX1glhfMEf5UIbI1W5h1/POmI+/f97S0Pmtv3sad+w+FjrPfGEiIqLM4GXOcLsVI7GophR5PipkxuL3rtfOAX/GSABATVkhipyjjQfGJjE87s+M5tlIRpTEVCac00SesRc4p3+JUWQeEJG/wrzpvQjAPclbIhEly/G+Eew60ovg9MkxnrjNHjq3eekUtyRKuecAzOTV2LHpbqCqkyLyIwCbAVwG4BvOVW5HcLx2UPfy3hmshzJcrnYL2z76stW4d28n2k6NoG9kAl/+U/izjYuWM0aCiIgoEzTVlGInzGzldOcMZ1K+MBDZMdzlw5xcuxPXT4PnAEBEUF9ZHPrwobN/DMvqUl1STY+U/RQiUgDg752zsYbdRDvLOd0f5/oDMIXhVZimMMz8Q6L0O9Y7gpd9/QH0j/r/k7MFVSV4wVn1Xi+DcpiqXpGiu+5yTkNH16jqkIi0A2gSkYUxcobdVtF4+1/KMrneLewqLy7Al69ci+t+tB2AySd0XXQGC8NERESZwD4K1O7gTYe2iHxh/xeG6yv93TFsr6nRZx3DgFmTWxju6B/FsrrsCDRI5TFyXwRwLoA7VfXPCdyeGYhEGewXjxzJiKIwALxp8xIUcOgcZaeLnNODUZff65y+LMY2L4+6DWU5dguHXXxmHd580ZKIy0y+cO4+J0RERJmkySrItnkYJdFU4/+YwsiMYR8Whq0uZnutfhHx/Pmw43q2UtIxLCLvBfBBmLzBt6TiMabiZB2exukk3pDm5RBlvYlAELc+2hY6f+nKOlSVFnq4ovjOqK/AOy8/w+tlEM2aiGwA8Hh07JKIXAHgA87ZX0Rt9j2Y/fEnReQ3qnrK2WYZgP8HYAzAT1K5bvIHdguf7mMvPxv37e0KdYBsWFKb9onmRERENDt2p25bujuGreiKTOgY9vvwObtY7cdBfvbz1+nDwvpsJb0wLCLvhsk1fBrAFarak+CmzEAkylB3Pd2B7kGzY1lQVYKfXL+RHblEqfPfAFaKyMMA3E9k1gJ4ofP9p1T1YXsDVX1YRP4bwL8C2CMitwMoAnANgHkA3qOqh9OxePIWu4VPV1FcgK9ctRZv++lOjE0G8cZNS6bfiIiIiHzBzvZNd5SEPeyuKQMKw3axtWtgDKoKEf8MzOvyecdwg8+H981WUgvDIvJ+AF8D8CRMUbhzBpvvc05XxbmeGYhEPnXj9qOh7/9u42IWhYlS6+cAXg9gI0wMRCGADgC3Avi2qj4YayNV/aCIPAHTIfxPAIIAdgH4iqr+Ph0LJ2+xWzi+i8+sw10fuBxD45M4e2GV18shIiKiBC2yCsMn+kcxGQim7f2o3aGcCR3D5cUFqCwuwMDYJMYDQfQOT6C2vMjrZYVEdAz7MGPYHojXySiJ04nIR2FyhR8H8GJV7Z7hXdznnL5ERPLsQ2RFpBLA8wEMA3gkCcsloiQ53D2Eh541/93zBLh242KPV0SU3VT1xwB+PMttfwrgp8lcD2UOdgtPbcl8/2cDEhERUaSSwnzUVxaja2AMgaDiRP9oxEC6VBmdCIQ6XPPzBAt82OEaS31VMQa6zGygjoFRnxWG/d0x3JilHcNJ+RhFRD4FUxR+DKZTOG5RWEQKRWS1iESEfKrqcwD+AmAZTDeT7bMwE9Z/rqpDyVgzESXHTTvD3cIvOKsh4hNbIiLyB3YLExERUbbyImf4mBUjsaCqJGOOmm2s9GfO8OhEAH0jEwCAgjzBvDL/FKxdkRnD/nnu5mrOHcMi8lYA/w4gAOBBAO+NkVFy2OlSAoAmAM8AOAJTBLb9C4CHAXzTGaLzDIDNAF4AEyHxybmul4iSZ3wyiNutoXPXbWYuIxGRH7FbmIiIiLJVU00pWo72AkhfznCm5Qu77K5XPw1Qs/OF6yuLkZfnn+xjV2RR3T/P3VwlI0piuXOaD+D9cW5zPxI4dFVVnxORC2EKzS8D8AoAx2GG2X3WnaJORP7w56dO4OTQOABgUXUJtpzV4PGKiIgoGruFiYiIKJvZ0RHp6hjOtHxhV0TXq49yciPzhf0XIwEAVaUFKCrIw/hkEEPjAQyOTaKiOKmj2zwx559AVbcC2DqD2x8GELf0r6qtAN4213URUerZQ+eu2bgE+T78VI+IKNexW5iIiIiymd2x2947nJbHtDuTmzMoTrG+0p85uRH5wpX+GzwHACKCxqpitPaY331n/ygq6is8XtXcZUYIChH5zsGuQfzt4EkAJmz/Gg6dIyLynf/f3p3Hx3mX997/Xtp3eZWc2E5iZyVAyGo7JBCS0DTslCaFJKWUFloKlKVQykMXQp/DoT1PDzuFthzIKY0hhUBoypaShZAS29kTsieOE1teJO/at7meP+6R9BtZkkea5Z6Z+/N+vfQazaqf7jj6Sddc9/eiWxgAAFS6ODKGdxyYKkAXY9hdvoQdw6VVGJ5aSykOnptQqhnNuaAwDGBBvr1lqlv4ktM6tKK9dH94A0BS0S0MAAAqXdixG2b/FlL5ZgyXZpREuJaOEu0YlqYfv9IprOeCwjCAeRsaHdf37mPoHACUMrqFAQBAEoSF2Z0HB5VKecG/ZvlmDIfD50qoMFwmHcNhFEcpHb9cUBgGMG8/e3S3DgyMSoomwL7y5OUxrwgAMB3dwgAAIAma6mq0pLlOkjQ67gXvhB0ZS01GH5hJx7SXT2G4ozWz47UYRfRs7OkNh8+VR8dwKUVx5ILCMIB5uz4YOnfVutUMnQOAEkO3MAAASJKVi8Kc4cIOoNt9aEgT9dTO1gbV1ZRPaa2xrlqtDTWSoiL6gYGRmFcUyRg+V8Idw2HH9Z4SiuLIRfn86wVQEp7p7tWW5/ZLkmqqTL9zLkPnAKDU0C0MAACSJIxzKHTO8I6DU4XncsoXnlCKOcNhlES5ZAzTMQwgkTZu3j75+atf1KmOEn43DwCSiG5hAACQNJkdwwUuDAevH37dcpHR9VoCxc3BkXEdHhqTJNVWmxY31cW8otmFReueEimq54rCMICsRUPnpgrDDJ0DgNJDtzAAAEiasGO40IXhrjIdPDchI2e4BAaodYf5wq0NqirhqMqOaR3D7qWR0ZwLCsMAsvajh3dNvpN33JImXXjSsphXBAAI0S0MAACSaOXipsnPC50xnNExXI6F4RLrGA7zhUt58JwktTXUqKE2KqUOjIyrb3gs5hXljsIwgKxt3DI1dO5t61aX9Dt5AJBEdAsDAIAkKmbGcFeQMbwqKEiXi86gY3hPbykUhssjX1iSzGxaznD8Hde5ojAMICtP7u7Vfc8fkBQNnbvyHIbOAUApoVsYAAAkVdi523VgsKCn+Jd/xnCpRUlMraGzDGYYhcXr7hLouM4VhWEAWdm4+fnJz3/zxSu0vMTfyQOAJLl323597MaHJq/TLQwAAJKkraFWbQ01kqThsZT29o0U5OuMjae0+9BUMbAcM4Yzhs+VwAC1sLhaFoXhsLBeAscvVzVxLwBA6RscGdf3H+iavH4NQ+cAoCRs2rpPX7z1af3q2X0Zt9MtDAAAkmbl4iYd3nVYUpQzXIhmpj29wxpLRd3Iy1rq1FBbnfevUWiZw+fi73gtpygJaVoURwkcv1xRGAZwVDc/vFO96aFza5Y16/wTl8a8IgBILnfX3c/u0+dvfVpbntufcV+VSR+57FS6hQEAQOKsWtyox9OF4a6DgzrruMV5/xpdGYPnyi9fWMoc8NbTO6xUymOdH5Q5fK70O4YzOq5LIIojVxSGARzVxs1TQ+euWrdaZgydA4Bic3f98um9+uKtT+vedOb7hOoq05vPXKn3X3KS1ixrjmmFAAAA8QnzfsMc4HzacSAYPFeG+cKS1FBbrfbGWh0aHNVYyrV/YETLWuLr1O3uDaMkyqBjuK20hvflisIwgDk9tvOwHtx+UJJUV12lKxg6BwBF5e6648kefeHWpyd/Hk+oqTL99tmr9N6LT9TxSykIAwCA5Fo1bQBdIYSvW475whM62+p1aHBUUhSHEGthOOi6DWMaSlUYd9FDxzCASrdxy9TQuctfskJLmutiXA0AJIe76+ePd+uLtz6tR7oOZdxXW2268tzV+pOLTtTqJeV5GiMAAEA+hYXasLM3n3ZkREmUb2G4o7VBT+3pkxQVZl98bDzr6B8eU+9wFFtZV12lRU218SxkHjroGAaQFP3DY7rpgZ2T169m6BwAFFwq5brlsd364q3P6LF0Tt6EuuoqvW3dar3nohN1bJmevggAAFAIq4LM366DBeoYPlgZHcMdGTm58RU3u3vDfOH6soit7Jx27Ny9LNY9GwrDAGZ180M71Zd+9+7E5c1av2ZJzCsCgMo1nnL95Ne79KVbn9GTe3oz7quvqdLV64/TH7/yRK1oL/1T7AAAAIptesZwIQp2YSfyykXle9ZWmJMbFmeLrTsoSocRDaWspb5GTXXVGhgZ19BoSoeHxtTeWPqdzrOhMAxgVhu3hEPnjivrd8EAoFSNp1z/+fBOfem2Z/RMd1/GfQ21Vfrd9cfrjy5aq44yyFwDAACIy6KmWjXXVat/ZFwDI+M6ODCqxXmMQkylXDsPThUyyzlKorO1NDqG9wRF6bBYXcrMTB2t9dq2L3qToKd3iMIwgMrz665DenhHlGlZV1OlK85ZFfOKAKCyjI2n9B8P7dSXb3tGW/f2Z9zXVFett59/vN79irWxDgMBAAAoF2amlYsbJ7NzdxwYzGthuKdvWCPjKUlREbqlvnxLahk5uTEOUAs7hsulMCxFx2+iMLzn8LBO6miNeUULV77/igEU1PWbp7qFX/fSY7SoiaFzAJAPo+Mp/eCBLn3l9mf0/L7MwSgt9TV6x8uP1x9euJZhnwAAAPO0anHTZGG46+CAXrqqPW+vHQ6eK+d8YSkzJ7c7xgFqYbdymHtc6jozCuvlPYCOwjCAI/QNj+k/HuyavM7QOQDI3chYSjfev0Nfuf2ZjD8sJKm1oUbvvGCN/uCCE3gjDgAAYIGm5wznU2a+cHkXhsOIsu44O4bD4XNlFJuWGcUR3/HLBwrDAI7wwwe71D8yLkk6uaNF5x6/OOYVAUD5Gh4b13fv3aGv3vHsEROy2xtr9YcXrtE7Xn5CWWeTAQAAlIKwkzffheHw97hVi8t38JyU2Z3b0zes8ZSruqr4M4X2ZERJlE/HcEeJdFznA4VhABncXddvmoqRuHo9Q+cAYCGGRsd1wz3b9dU7ntXuaaeYLWqq1btfsVa/d/7xam2gIAwAAJAPKwtYGA5fr9w7hutrqrWoqVYHB0Y1nnLt6x+OpWM37FYup4zhcK1xdlznA4VhABke2nFIj+06LEmqr6nSW85i6BwAzMfgyLg2bnlB//SLZzNOj5Okpc11evcr1+p3Nxxf1gNLAAAASlHYyTv9TK1cdVVQxrAkdbY26ODAqKSouBlHYTijY7iMoiTCY0XGMICKsnHz85Ofv/6MY9XeRCcbAGRjYGRM/7bpef3znVu1t28k475lLfV6z0VrdfX649RUx69fAAAAhZCZMTwwxyPnLyNjuAIKwx1t9XpyT6+kiTiE/A3qy0bf8NhkhGVdTZXaGsvnd+Qw9mIPURIAKsXhoVHd/NCuyesMnQOAo+sbHtO/3r1NX//lc9rfn1kQ7myr13suOlFXrTtODbXVMa0QAAAgGZa11Km+pkrDYyn1Do3p0OBoXuY4uHtFZQxLmXEIcQxQ656WL1xOEZYd046du5fV+kMUhgFMuumBLg2ORu/YnbaiVWcftyjeBQFACTs8NKp//dU2ff2u5yZPw5twTHuD3vuqE3XluaspCAMAABSJmWnl4kZt7emXFMU/5KMwvK9/REOjKUlSa31NRQwNzuh6jSEOISxGl1OMhCS11Neoua5a/SPjGhlL6fDgWNmebU1hGICk6B3QjZsZOgcAR3NoYFTf/NVz+sZdz+nw0FjGfSsXNep9F5+k3z5npeprKAgDAAAU26rFTVOF4YODOv3YtpxfM8wXroQYCWl6Tm4MHcO9YcdweRWGpWjNW/dG/8729A5RGAZQ3u5/4aCe2B3lCzXWVuvNZ62MeUUAZmJm10l6x1Eedpu7Xxo85/clfXOOx/+Ju38t99VVtsGRcf3jHc/ouv/ept7hzILw6iWNev/FJ+m3zlqlupqqmFYIAACAMGe4K085w5kxEpVRGA47hrtj6Rie+prLW+vneGRp6mirnyoMHx7SKZ2tMa9oYSgMA5CkjG7hN7zsGLU1lOe7XUAC3CRp2yz3vV3SWkk/meX+H0p6cIbb7811UUnwqZsf1Xfu2Z5x2wlLm/T+S07Wm848VrXVFIQBAADiFhZudwSdvrkIB89VQr6wlJmT290bR8ZwECVRph3DE+LouM4XCsMAdGhgVP/58M7J69esPz7G1QCYi7vfpKg4nMHMFkn6mKQRSdfN8vSb3H22+zCHVMp1y2N7Jq+fuLxZf3rJyXr9GceohoIwAABAyQgLw2Gnby4yoiQWVUrHcFjYjKFjuDcsDJdhx3DQ5RzGYpQbCsMA9P0Hdmh4LArSf/GxbTpjVXvMKwKwAG+X1CjpO+6+N+7FVJqnunu1v39EkrS4qVa3fPgiVVeRww4AAFBqCtMxXHlREstbpgqbe/uGNTaeKmrDQ1iMLveO4W46hgGUK4bOARXj3enLf57jMWea2YckNUjqknS7u+8o9MIqwd3P7pv8fMPapRSFAQAAStTKRVNRD3nrGD5YecPn6mqqtKS5Tvv7R5RyaV//SFELtGGucUdZZgzH23GdL3kpDJvZFZIuknSmpJdJapV0vbv/7jxfZ5uk2c5h3+PuK3JYJoAZ3Pv8AT3d3SdJaq6r1pvOZOgcUG7M7HxJL5X0lLvfPsdDPzjt+riZfV3Sh9w9q99mzOy+We46LZvnl6uwMHz+iUtjXAkAAADm0tFar9pq0+i4a3//iAZGxtRUt/Dyl7tP6xiujIxhKTpWE2fFdR8eLlph2N0zco07yrFjOChmJ74wLOmvFBWE+yTtUG5/HB6S9PkZbu/L4TUBzCLsFn7jmSvVUs+JBEAZ+qP05b/Mcv9zkv5U0i2K9ul2SRdK+oykP5bUJunqAq+xbKVSrs3P7Z+8fv5aCsMAAAClqqrKdOyiRj2/LxoY13VgUCd3ti749Q4PjqlveEyS1FhbrcVNlTOovbOtQU/s7pUUFTdfquLESvYNj2lgZFyS1FBbpbaG8qtDxD28L1/ydeQ/rOgPzWcUdQ7P1a10NAfd/dp8LArA3A70j+hHj+yavH7N+uNiXA2QHEc5Q2Yms56FY2btkn5Hcwydc/dfSPpFcNOApO+a2SZJD0m6ysz+3t0fOtpC3P2cWdZxn6Szj/b8cvTYrsM6NDgqSVrWUq+TOlpiXhEAAADmsmrxVGF4R46F4e0HBjJet5KiF8MIhz1FHKC253A4eK6hLI9pxvC5w8Ny97L8PvJSGA5PWy3HgwAk1Y3379BIeujcGava9ZKVDJ0DiuRZSfP5zWvnHPf9rqQmLWDonLtvN7MfS7pG0isVFYkxzaatYb7wEn7XAQAAKHErFwUD6HLMGa7EfOEJnRk5ucXreg3zhTtbyy9GQpKa62vUWl+j3uExjYyndHBgVIub6+Je1ryVYq92vZn9rqTjJPVLeljSne4+Hu+ygMri7tq4JRg6t45uYaBY3P3SPL7cxNC5f1rg83vSl815WEtFCgvD5AsDQOUzs7+XdK6kUyQtkzQo6XlJN0n6srvvm+E5L1cUsbhBUqOkpyV9Q9KX+FsWKL4wB3hH0PG7EJn5wpVWGJ7qeu0pYsdwGL2wvK38Bs9N6GirV29PFDOyp3eIwnCerJD0rWm3PWdm70yfCntUSR2MA8zH5uf2a2tPvySppb5Gb3jZsTGvCMB8mdl6RRn/T7n7HQt8mfXpy615WVSFGSdfGACS6MOS7pf0X5K6Fb15ukHStZL+yMw2uPv2iQeb2Zsk3ajobKAbJO2X9AZJn5N0gaQri7l4AJkdw10HcuwYDp6/clHlDJ6TMnNyi9kxvKcCOoYlqaO1Qc+m6yrdh4d12oqYF7QApVYY/qakX0p6VFKvpLWS3q9oqM5PzOz8bPIPARzd9cHQuTefdayaGToHlKOJoXP/PNeDzOxcd7932m1Vkv5C0vmS9kr6aUFWWOYe3XlIvUNRF0BnW73WLKOxGgASoM3dj2idM7NPS/qEpP9H0nvTt7UpGv46LulVE/utmf21pNskXWFmb3P37xRr8QAyO3t35FgY3jEtY7iSZGQMH44rY7h8O4bDtRfz+OVTSVWC3P1T0276taT3mFmfpI8oeof2t7J4ncQNxgHmY1/fsH7666mhc1evm88MLAClIP2H6FslDUv6v0d5+D1m9mtFGcJdktoVdTC9RNEgumvc/XABl1u27n42iJFYu5R8YQBIgJmKwmn/rqgwfHJw2xWSlkv61/BNWHcfMrO/knSrpD+RRGEYKKIwC7iLjOFZxZUxHA66C9dQbsK1h/EY5aQq7gVk6Wvpy1fGugqgQnzvvh0aHXdJ0pmrF+n0Y9tiXhGABbhG0amtP8hi6Nw/KDqt9RJJH5T0e5JqJX1F0kvd/ZZCLrSc3U2+MABgyhvSlw8Ht12SvpzpzJs7Fb0B+3IzK9+WOKAMrWhrUHVV9IZ+T++whkYXHvVdyRnDy4OO4X39wxobTxXl6/YEReiwa7ncZEZx0DFcSAzGAfIklXJ9Oxw6t56hc0A5cvevSvpqlo/98wIvpyKNjqd0T0a+8LIYVwMAKDYz+6ikFkVn2pwr6UJFReG/Cx52avryqenPd/cxM3tO0osVxSQ+fpSvx6wcIE9qqqu0oq1hstt358FBrV3eMu/X6R0a1aHBUUlSXU2VljWXbxFzJrXVVVrWUqe9fSNyl/b2jWhFe+E7eMOO4Y4y7hgOi9rdRey4zqdyKQxvSF8yGAfI0d1b92nbvigjqbWhRm84g6FzADCTR7oOqX8k6i5ZuahRq5dUVocIAOCoPiqpM7j+U0m/7+49wW3t6ctDs7zGxO2L8rs0AEezanHjZGF4x4GFFYbDGIlVixpVVVV5sWIdrQ3a2zciKep6LXRh2N0zh8+VdcZw0DHcW54dw0WPkjCzWjM7zcxOnHb7i8zsiI5gMztB0pfTV/+tCEsEKtrGYOjcW85aqca66hhXAwClK8wX3kC+MAAkjruvcHeTtELSWxR1/T5gZgWZW+Pu58z0IemJQnw9oNLlI2e460Dl5gtP6CjyALXDQ2MaGo0iK5rqqtVSXy49q0cKi9qJ7hg2szdLenP66or05flmdl36873u/tH05ysVnULzvKQTgpd5q6SPmNmd6ft6JZ0o6XWSGiT9WFFGIoAF6ukd1s8e3T15/er1DJ0DgNlsIl8YACDJ3fdI+oGZ3a8oMuJfFQ1wlaY6gttnem5w+8GCLRDAjFYtmirk7jgwsKDXqOR84QmdrcUdoNYTxki01pd180VHxrEbUirlZddVnq+y/JmS3jHttrXpDykq9H5Uc7tdUT7TWYompTcr2jzvkvQtSd9yd8/PcoFk+u592zWWiv43Ouf4xTp1RWvMKwKA0jQyltK92w5MXqcwDABw9+fN7DFJZ5rZsvTw1ycV5Q+fIikjI9jMaiStkTQmYhGBolu1uGny87Dzdz7CTuOViyq0MJzR9Vr4juE94eC5Ms4XlqTGumq1NdTo8NCYRsddBwZGtLSlvKIx8lIYdvdrJV2b5WO3STqifO7uv5D0i3ysB8CRUinXd7Zsn7x+9TqGzgHAbB7acVCD6enVxy1pqtg/BAAA8zYxoGM8fXmbpGskXS7p29Me+0pJTZLudPfyPMcYKGNh9MOOBRaGw07jyo2SCHJyixCHkJkvXN6FYSk6foeH+iRFHdflVhguesYwgHjc9cxevbA/2tTaG2v1ujOOiXlFAFC6MvOFl8S4EgBAMZnZKWZ2RCyEmVWZ2acldUj6lbtPnFbyPUl7Jb3NzM4NHt8g6X+kr361wMsGMINVec4YDjuQK0lHa5AxXIQBamHxubO1vIqoM+ksckZzvpVvwjOAeckYOnf2SjXUMnQOAGYTFoaJkQCARHmtpM+Y2V2SnpO0T1KnpIsURSXulvTuiQe7+2Eze7eiAvEdZvYdSfslvVFRVOL3JN1Q1O8AgCTpmPZGmUnu0u7DQxoZS6muZn79kWGncaWeQdYZY8dwOPiuXGVkNJfhADoKw0ACdB8e0n89vmfy+jXriZEAgNkMjY7rvheCfOG1y2JcDQCgyH4u6SRJFyqaf7NIUr+ioXPfkvRFd98fPsHdbzKziyT9paTfVjQ8/RlJf5Z+PLNygBjU1VSps7VBuw8PRcXhQ0M6bmn2Xb+DI+Pa1z8iSaqpsoqIPZhJ+H31FKFjuCcYcFcJxzQzioOOYQAl6N/v3a7x9NC5dWuW6KQOhs4BwGweeOGgRsZSkqQ1y5q1or38f2EFAGTH3X8t6f0LeN5/K+o2BlBCVi5u1O50sW7HgYF5FYa7Dk7lCx+zqEHVVUeMy6oIy1rqJjur9/aNaHQ8pdrqwiXPZnQMt5b/79lhFEd3b/l1DJMxDFS48ZTr28HQObqFAWBud28N84WJkQAAAChXYc7wjnnmDIcxEqsWVWa+sCTVVFdpafNUcbOnwMXNMMe4sxKiJMq8Y5jCMFDh7ny6ZzJof3FTrS5/yYqYVwQApW0T+cIAAAAVIcwFDgu92cjIF15cmfnCE4o1QM3dM3KMOyogSiLj2NExDKDUhEPnrjhnleprGDoHALMZHBnXA9un8oU3rF0S42oAAACQi1WLpzp9u+ZZGO4KOoxXVXxhOBigVsDi5uHBscnItua6arXUl3/Cbcaxo2MYQCnZdWhQtwZD565aR4wEAMzlvucPaHQ8ymQ/qaOlInLPAAAAkirs9N1xYGCORx4po2N4UaUXhoOc3AIWNzNjJCrj9+zlrZkxHKlUec0bpTAMVLAb7tmuiZ9J569dqrXLW+JdEACUuLu37p38/HzyhQEAAMpa2OnbNc+M4a6gkBx2Hlei5a1hTm7hOoYzBs9VQL6wJDXUVqu9sVaSNJZy7R8YiXlF80NhGKhQY+Mp3XDP1NC5qxk6BwBHdTf5wgAAABUj7PTddWhIY+OprJ+bMXyu4qMkipMxHBadK6VjWCre8SsECsNAhbrjyR7tOhT9QFraXKfffDFD5wBgLv3DY3p4x6HJ6xvoGAYAAChrDbXVWtYSFe3GU571cLDhsfHJrN0qk1a0V04RcyadrcXJGO4OoiQ6WiujY1ianjNcXgPoKAwDFWrjlmDo3LmrVFfD/+4AMJd7nz+gsXT+zmkrWrWkuS7mFQEAACBXGTnD+7PLGd55cKqAuaKtQbXVlf33dFjYLGTHa3eFdgyHOcNh8bscVPa/bCChug4O6o4nuyevX3UeMRIAcDRhjATdwgAAAJVhITnDXRkxEpWdLyxNGz5XwI7hzIzhyikMZxbW6RgGELMbtrwwOXTuwpOW6YRlzfEuCADKwN1byRcGAACoNKuCnOEwN3guO4LBcysrPF9Ykpa21KvKos/3949oZCz7LOb5CAvDnZUUJdFKxjCAEjE2ntIN9zJ0DgDmo3doVL/uivKFzaQNaygMAwAAVIKMjuEsC8NhZ3GlD56TpOoqm8xilqSevsJ0vYbdyHQMlwYKw0CFufWJ7skfRMta6vUbp3fGvCIAKH33bNuv8fSpFqcf06b2ptqYVwQAAIB8yMgYPphdxnDYWbxyUeUXhqXC5wy7e0bGcCUNnwuL3GQMA4jVxs1TQ+d+59xVFR+SDwD5EOYLn0++MAAAQMUIM4Kz7hhOWMawNC1nuACF4YMDoxoZjyIqWutr1Fxfk/evEZewyN1NxzCAuGzfP6A7n+6RFJ0KfdU6YiQAIBvkCwMAAFSmsON358EhpSYG8swhaRnDkrS8tbBxCHt6w8FzldMtLGV+Pz19w5NnIpYDCsNABfnOPS/I0z9/XnHycq1ekox3NgEgF4cGRvXozsOSpCqTzluzJOYVAQAAIF+a62u0OB0TNjKeOmp+7uh4SruDjtljF1VOFu5cwo7hQkRJ7MmIkaisY1pfUz35b2w85drXXz5dwxSGgQoxOp7Sv9+7Y/L61XQLA0BWNj+3b/JNtZeubFdbA/nCAAAAlSQjZ/jA3DnDuw8NaaLhs6O1XvU11YVcWsnozMjJzX9hM4yn6KywjmFp2vErozgJCsNAhfj5Y3vUk/7h3dFar0tf1BHzigCgPIQxEhuIkQAAAKg4qxZNnU274yg5wzsy8oWTESMhFb5jOCw2h0XUSrE8zBkuowF0FIaBCrFxy9TQubeet5qhcwCQJQbPAQAAVLawY7jr4NyF4fD+lQkZPCdlxjsUouM1LDZ3VGBhOCx2FyKjuVCoHAEV4Pl9/frl03slRUPn3nre6phXBADlYX//iJ7Y3StJqqkynXcC+cIAAACVZlVGlMTROoanoiaS1DEcDlDbU4CO14zCcGslRkkUtuO6UCgMAxXg21u2T35+8akdWpWgdzUBIBebgxiJM1a1q7m+JsbVAAAAoBBWLgo6ho9SGA7vD59X6ZY216u6yiRJBwdGNTw2ntfXr/QoCTqGAcRiZCyl7947VRhm6BwAZC/MFz6ffGEAAICKFDZPHW34XFIzhqurTMtbgpzcPBc3w9erxOFzYRd0DxnDAIrlZ4/u1r7+EUnSMe0NetWpy2NeEQCUjzBfeAP5wgAAABVpesawu8/62DBjOEmFYSkzTiKfA9RSKc94vTDPuFJ00DEMIA4bN2cOnath6BwAZKWnd1hPd/dJkmqrTeceT74wAABAJWpvrFVrQxQZNjSammyumm485doZDp9blKyYxrBgm8/i5oGBEY2OR8X4toYaNdZV5+21S0VmlAQdwwCKYGtP3+Rp0FUMnQOAedkUxEicuXpRRf6CCgAAgEg2OcPdvUMaS0UFzKXNdYn7/TCMeOjOY3EzzBfuqMB8YUkZMRx7+4Y1npq9K72UUBgGyti3t0x1C19yWqeOaU/WaS4AkIuMfGFiJAAAACpaZs7wzIXhpOYLT8joeu3NX8dw2EFbifnCklRXU6UlzXWSpJRL+/rKI06CwjBQpoZGx/W9+3ZMXr9mPUPnAGA+NoX5wgyeAwAAqGirMnKGZx5AF3YSr0xkYXiqaJvPOISMwXMVmC88IRxAVy45wxSGgTL1s0d368DAqKTolJhXnsLQOaDcmFmtmX3QzL5pZg+a2YiZuZm9K4vnvsPMtphZn5kdMrM7zOz1czy+2sw+bGYPm9mgme03sx+b2cvz+12Vhz2Hh7R1b7+k6N39s49bHPOKAAAAUEhhYXj2juGpgnHYYZwUYcZwdx4Lm2GRuVKjJKTyzBmmMAyUqeuDoXNvO2+1qqssxtUAWKBmSZ+X9PuSVkjanc2TzOwfJF0n6RhJ/yLp3yS9VNLNZvb+GR5vkr4j6bOS6iR9WdIPJL1S0p1m9qbcvo3yc3fQLXz2cYvUUJus/DgAAICkySZjuCtj8FzyOoY7wozh3gJlDLdWZpSENK3jOo/Hr5AoDANl6JnuXm15br8kqbrK9DsMnQPK1YCk10o61t1XSPrG0Z6Q7vD9iKRnJZ3h7h929/dJOkfSfkn/YGYnTHva2yRdIelXks509z939z+UdLGkcUn/YmatefqeykJYGD5/7bIYVwIAAIBiIGP46DI7XguVMVy5HcNhxzVREgAKZuPm7ZOfv/pFHRX9gxWoZO4+4u4/cfdd83jae9KXn3b3A8FrbZP0FUn1kt457Tl/kr78K3cfCp5zj6QbJC1XVDhOjIzBc+QLAwAAVLyVGRnDg3L3Ix6T9IzhJU11qkmfjXxocFRDo+N5ed1wkF2lDp+TMr+3niR1DJvZFWb2JTP7pZkdTucj/tsCX2uVmX3DzHaa2bCZbTOzz5sZ4X+AoqFzN94/NXTu6vXHx7gaADG4JH350xnu+8m0x8jMGiS9XFF38i+zec5czOy+mT4knZbV6ktA18FBvbA/yo9rqK3Sy1a3x7wiAAAAFNriplo11UXxYX3DYzo0OJpxfyrl2pHwKImqKtPyIOohXznD3UnpGC5Qx3Uh5atj+K8kvV/SmZK6FvoiZnaipPsUdTptkfQ5SVslfVDS3WZGSw8S78eP7JrcwFYvadQrTuIUaCApzKxZ0kpJfbN0GT+dvjwluO1ESdWStrr7WJbPqWibghiJc49fovoa8oUBAAAqnZllFHunx0ns7R/WyFhKktTeWKvWhtqirq9UhMXNfOQMp1KunqBjeHlFZwwnd/jchxX9QdmmqdNVF+IfJXVI+oC7v9ndP+7ulygqEJ8q6dM5rxQocxszhs4dpyqGzgFJMtHaemiW+yduX5Tjc2bl7ufM9CHpiWyeXwqIkQAAAEimMDd4emE46fnCEzqDwm0+ul73D4xoLBXFdrQ31lb00OeM4XNJ6hh299vd/WmfKaAlS+lu4cskbVOUkRj6pKR+SW9Pd0sBifTUnl7d+3wUKVpTZbry3FUxrwhAOvLI5/GxoKgl5E84eG7DWgrDAAAASTE9ZziUkS+cwBiJCfnues0cPFe53cKStKylXpbu3dvXP6yx8VS8C8pCTdwLCFycvrzF3TOOnLv3mtl/Kyocb5B0a7EXh2Q40D+i7z/QdUTWUKm4d9v+yc8ve3FnxsRLALF5VtJ8fmPamcPXmujunS0Ud+L2gzk+p2Jt3z8w+UdAU121zlhFvjAAAEBSrFrcNPn5jgMDGfdldgw3Kak6wo7hPERJhDnFlZwvLEm11VVa2lynvX0jcpf29o1oRXtpf8+lVBg+NX351Cz3P62oMHyKjlIYTg/BmUnZDMZBPD54w4O686meuJeRlavXMXQOKAXufmkRv1a/mXVJWmlmx8yQM3xy+jLcS5+VNC5prZnVzJAzPNNzKlbYLXzeCUtUW52vVC0AAACUurATuGtalETXwalC8cokR0mEGcN5iEMIO4YrOV94Qkdrg/b2jUiKvvdSLwyX0l9Dec1ABObr6T29ZVMUPm1Fq15OLiaQVLelLy+f4b7XTHuM3H1I0q8kNUl6RTbPqWTkCwMAACTXSjKGj6ojiHvIx/C57t7kdAxL03OGS38AXSl1DOdNegjOEdKdxGcXeTkoE9cHQ91etqpdF5/WEeNqZtdcV6M3vOxYhs4ByfU1SW+X9JdmdpO7H5AkMztB0vskDUv65rTnfFVRUfh/mNml6WKxzOw8SW+V1CPpxuIsPz7untExfD75wgAAAImyiozho8rMGM5vx3BnQjqGJ4RF8VJVSoVhMhARm6HRcX3//h2T1z9y2al65SnLY1wRgKQws49rKurozPTlO83swvTnd7n71yce7+6/MrPPSvozSQ+b2fck1Skq8C6R9Kfuvm3al/mOpLdIukLSA2Z2s6Sl6edUS3q3ux/O9/dWarbtG9Du9C+mrfU1evGxbTGvCAAAAMW0rLledTVVGhlL6dDgqHqHRtXaUCt3p2M4LSNjOC/D55LbMdxNx/C8PJm+PGWW+xOVgYji+s+Hd+nwUBS7edySJl140rKYVwQgQS6XdNG0216e/pjw9fBOd/+ImT2iqEP4jySlJN0v6f9z9/+c/gXc3c3sKkWREn8g6U8VDcy7U9L/cPdf5el7KWlht/C6NUtUQ74wAABAolRVmVYtatTWvf2Soq7h01bU6sDAqAZHxyVJLfU1am+sjXOZsVrcVKfaatPouKt3aEyDI+NqrKte8OuFcRRhTEWl6shzx3WhlVJh+Pb05WVmVuXuqYk7zKxV0gWSBiRtimNxqGwbNz8/+flV644jpgFA0bj7qxb4vOskXTePx49J+lz6I5HIFwYAAMDKxVOF4R37B3XaijbtOBAMnlvUKLPk1gSqqkwdrQ2TURvdvUM6fmnzgl8vHGAXxixUqowojjxkNBda0VtlzKzWzE4zsxPD2939WUm3SDpBUQdU6FOSmiV9y937i7JQJMYTuw/r/hcOSpJqq01Xnrsq3gUBAPJuer7wBvKFAQAAEmmmnOEuYiQydGQMUFt41+t4ytXTFxSGk9Ax3BpGSSSkY9jM3izpzemrK9KX55vZdenP97r7R9Ofr5T0uKTnFRWBQ+9VdJrrF83s0vTj1ku6WFGExF/mY71AaGMwdO6yF6/QspbK/0EFAEnzbE+f9qZ/KW1vrNXpx5AvDAAAkEThYLmJTuEwX3glhWF1toZxCAvvet3XP6zxlEuSFjfVqr5m4ZEU5SLsGO4ug47hfEVJnCnpHdNuW5v+kKIi8Ed1FO7+rJmdK+lvFWUuvlbSLklfkPSpicnrQL4MjIzpB/d3TV6/Zt1xMa4GAFAoYbfw+jVLiAwCAABIqFWLmyY/n+wYPkjHcCizY3jhxc3uhA2ek6RlLXUyk9ylvX0jGh1PqbaEZ5vkpTDs7tdKujbLx26TNOtfY+6+XdI787Eu4Gj+86Fd6h2Ohs6tWdZM5iQAVCjyhQEAACBldgRPdApnZgw3HfGcpAmLuD29C49DCDtml7cm4+zsmuoqLWupnzxuPb3DOnZR6b7ZULola6AIrt8yFSNx1brViQ6YB4BKlUq5Nm3dP3mdwjAAAEByZWQMTxaG6RgOhTm5uXQM70lgx7A0LWc4h8J6MVAYRmI9uvOQHtp+UJJUV12lK85ZHe+CAAAF8VR3r/b3j0iKss1O6WiNeUUAAACIS0drg2rSsWL7+kc0MDKWMXyOjOHMIm4uw+fConJnAgbPTcg8fqWdM0xhGIkVDp27/CUrtKS5LsbVAAAKJcwX3rB2KfnCAAAACVZdZRmn9j++q3cyYrKhtkpLqQ1kZgznMEAtqR3DYRG8m8IwUHr6h8f0wwd3Tl6/ej1D5wCgUoWFYWIkAAAAsDIoDG9+bl/G7URMSp2tQcZwDh3DPUFRuSMhGcNS1JU+IZeO62KgMIxE+o+Hdqov/Y7gicubtX7NkphXBAAohFTKtfm5IF94LYVhAACApAtzhDcHsyhWLWbwnCQtaqpVXXVUMuwdHlN/un4yX2FRtCNRHcNESQAlLYyRuGrdcbwjCAAV6rFdh3VocFSStKylXid1tMS8IgAAAMQtzBG+d9v+GW9PMjPLiJNY6AC1zIzh5BSGGT4HlLBHdhzSI12HJEl1NVW64pxVMa8IAFAom7aG+cJLeCMQAAAAGZ3B/SPjwe0UhieExc2FdL2Op1x7+6aKostbkhMlQccwUMI2bnl+8vPXvfQYLWoiWB4AKlVYGCZfGAAAAFJmxnA2tydRWNxcSNfrvr5hpTz6fElznepqklOC7MxDt3WxJOe/CiCpd2iUoXMAkBDj5AsDAABgBrN1BpMxPCWjMLyArteMfOEEDZ6TpKUt9apKn6i4v39EI2OpeBc0BwrDSJQfPrhTA+nTRE7uaNG5xy+OeUUAgEJ5dOch9Q5FgzI62+q1ZllzzCsCAABAKVjR3jBZuAsRJTElzBheSBxCUvOFJam6yrQsiM7o6SvdrmEKw0gMd88YOnf1eobOAUAlu/vZIEZi7VJ+5gMAAECSVFtdpWPaM4vAddVVicrBPZqO1jAnd/6FzT29YWE4ece1XHKGKQwjMR7acUiP7TosSaqvqdJbzmLoHABUsrvJFwYAAMAspucJH7uoQVUztREnVGfOHcNhlESyOoalaTnDFIaB+F2/aWro3Btedqzam2pjXA0AoJBGx1O6JyNfeFmMqwEAlAszW2pm7zKzH5jZM2Y2aGaHzOwuM/tDMzvib2gzqzez95nZFjPba2Z9Zva4mX3RzI6P4/sAcHTTYyPIF84Udrz2LGCAWk/CO4Y72nLruC6WmrgXABTDocFR3fwwQ+cAICke6Tqk/nSm/MpFjVq9hLw4AEBWrpT0VUm7JN0u6QVJnZLeIunrkl5jZle6u0uSmdVIulXSBZKekPRtScOSzpP0p5J+z8xe7u6PFfsbATC3ldMKw9M7iJOuszW3KISMjuGEZQxLmQP3untLt2OYwjAS4aYHujQ0Gk2BPG1Fq85avSjeBQEACirMF95AvjAAIHtPSXqjpB+5++QYeTP7hKQtkn5bUZH4xvRdv6WoKHyrpMumPedTkv5G0kcl/UFRVg8ga0d2DFMYDrU11qi+pkrDYyn1j4yrb3hMLfXZlxGTPHxOmp4xXLodw0RJoOJNHzp3DUPnAKDibSJfGACwAO5+m7vfHBZ407fvlvS19NVXBXetTV/+aPpzJP0wfbk87wsFkLOVizKjI6Z3ECedmakjh5zhzIzh5EVJ5JrRXCwUhlHx7n/hgJ7c0ytJaqyt1pvOWhnzigAAhTQyltK92w5MXqcwDADIk9H05Vhw26Ppy9fMkD/8+vTlzwu6KgALQsbw0YVxEt3z6HodG09pX//U45cnsDDcscBjV2xESaDiXR90C7/xZceqrYGhcwBQyR7acVCDo1G+8HFLmsiLAwDkLJ0l/Hvpqz8N7vqRpO8ripd4xMx+LmlE0jmSLpT0JUlfyfJr3DfLXactZM0A5nbMosx4AzqGjxTGIcwnJ3dv34iiJHZpWUudaquT15cadluTMQzE5NDAqH708K7J6wydA4DKF+YLn7+WbmEAQF78naSXSPqxu/9s4kZ3dzO7QtInJf2VpNOD59wqaaO7jwlAyamvqdaGtUu0aet+ndTRomMSmIN7NAuNkggfG3bOJsnS5npVV5nGU64DA6MaHhtXfU113Ms6AoVhVLQb79+h4bEo6uvFx7bpjFXtMa8IAFBoGYVhYiQAADkysw9I+oikJyS9fdp9DZL+VdJrJL1PUa7wgKKBdF+UdKeZXenuP9RRuPs5s3z9+ySdncv3AGBmX73mHN3xVLdefuIyVVUxi2i6sKg7nwFqmYPnkhcjIUnVVablLfXanT4W3YeHtXpJ6cWVJK+XG4nh7tq4ZSpG4mqGzgFAxRsaHdd9L5AvDADIDzN7v6QvSHpM0sXuvn/aQz4u6UpJf+nu/+Tuu939sLv/RNIVkmrTzwdQghY31+m3zlqVEZmAKZ0ZcQjZF4bDxya1Y1iafvxKM06CwjAq1j3bDuiZ7j5JUnNdtd50JkPnAKDSPfDCQY2kzxRZu6yZX/IBAAtmZh9SlBH8a0VF4d0zPGxiwNzt0+9w94ckHZB0vJnxTiWAshP+Lj2fKIluOoYlSR1tC+u4LiYKw6hYGzc/P/n5G89cqZZ6klMAoNLdvXUqRmID3cIAgAUys7+Q9DlJDyoqCnfP8tCJisfyGV6jXlJr+upIvtcIAIWW0fE6r4zhoGM4wY0aHa0LO37FRGEYFelA/4h+/OupN/SvYegcACTCJgbPAQByZGZ/rWjY3H2SLnX3vXM8/Jfpy0+kC8GhaxXN9bnH3XvzvlAAKLDl0zKG3T2r5+3pDTuGk1sYzui4nkcURzHRQomKdOP9OyZPJT5jVbtespKhcwBQ6QZHxvXA9ql84Q0UhgEA82Rm75D0t5LGFRV9PzDDnJJt7n5d+vNPS3qDpEslPWFmP5U0qGj43Lr05x8s/MoBIP/aGmrUUFulodGUBkfH1Tc8ptaG2qM+rzvsGG5NbpRE2HE9nyiOYqIwjIpzxNC5dXQLA0AS3Pf8AY2OR10MJ3e0aHmCfwkFACzYmvRltaQPzfKYX0i6TpLcvcvMzpb0F5JeJ+mdis7M3ZV+zN+7+xOFWy4AFI6ZqbOtQc/vG5AUdQ1nVRimY1hSZoxGNxnDQHFs2rpfW3v6JUkt9TV6w8uOjXlFAIBiuHvr1Jm+dAsDABbC3a91dzvKx6umPafH3T/q7i9y9wZ3r3P34939nRSFAZS7ztawuHn0rtfR8ZT29kWx6mbSspa6gq2t1GVkDPeWZscwhWFUnLBb+M1nHatmhs4BQCLcHeYLM3gOAAAAyNnyMA4hi+JmT5Clu6ylXjXVyS09ZmQM0zEMFN6+vmH99Ne7Jq9fve74GFcDACiW/uExPbzj0OR1OoYBAACA3GV2DB+9uNndS77whCVNdaqpinLqDw2Oamh0POYVHYnCMCrK9+7bMZkveebqRTr92LaYVwQAKIZ7tu3XWCr6+X/ailYtaU7uKWsAAABAvmQOUDt6YTgcspbkfGFJqqqyzDiJEuwapjCMipFKub4dxEhcs56hcwBKm5nVmtkHzeybZvagmY2YmZvZu+Z4zgVm9r/M7B4z6zGzYTN7zsy+bmYnzfKc69KvO9vHaYX7Lotj09b9k5/TLQwAAADkR0YcQhZREt0ZheFkdwxL0vJwAF0J5gwTvoqKcffWfdqWnpTZ2lCj15/B0DkAJa9Z0ufTn++RtFvS6qM850ZJyyX9StL1ksYknS/pDyW9zcx+w93vnuW5X5B0cIbb985wW1m5eyv5wgAAAEC+dbSFHa9HL2yGXcUdrcnuGJakztb5dVwXG4VhVIzrNz8/+flvn71KjXXVMa4GALIyIOm1kh50911mdq2kTx7lOZ+T9C133xneaGafkPRpSf8s6aWzPPfz7r4tpxWXoN6hUf26K8oXNpM2rKEwDAAAAORDWNydb5REBx3D0wbQlV7HMFESqAjdvUO65dE9k9evJkYCQBlw9xF3/4m77zr6oyef8/fTi8Jpfy9pUNJLzCxRldF7tu3XeDpf+PRj2tTeVBvzigAAAIDKEMZBdPcOyd3nfHw4fK6TjuHMjGaiJIDC+O69OyaHDp17/GKd0tka84oAoOhcUayEJM027vY1ZtaWvv8ZSbe5++FiLK6Q7n42iJEgXxgAAADIm5b6GjXVVWtgZFxDoykdHhpTe+PsjRgMn8sUdlz3VHKUhJmtkvS3ki6XtFTSLkk3SfqUux/I8jXukHTRHA9pdPfSK68jVqmU6zv3TA2do1sYQEJdKalV0iZ3PzjLY/5x2vVeM/t/3P0r2X4RM7tvlrtiG2BHvjAAAABQGGamzrYGPbe3X1KUMzxXYTijY5goiYw4jYrtGDazExUNwemQ9ENJT0haJ+mDki43swvcfd8cLzHdp2a5fWyW25Fgv3xmr7bvH5QktTfW6rUvPSbmFQFAcZnZGklfUrRP/tkMD7lT0o8lbZLULelYSb+lKM/4y2Y26u7/XKTl5tWhgVE9ujNqeq4y6bw1S2JeEQAAAFBZlrfWTxaG9xwe1smznKU9MpbS/v4RSdHv5ktbKAxnZgxXbsfwPyoqCn/A3b80caOZfVbShxUNw3lPti/m7tfmaV1IgI3Ths411DJ0DkDxmNk2ScfP4ynXu/vv5vHrd0j6iaTlkt7n7ndPf4y7f2PaTVsl/W8ze1LSzZI+bWb/x91ni6AIX+ucWdZxn6Sz57v+XG1+bp8mYs5eurJdbQ3kCwMAAAD5FBY3u+foeu3pmyp8LmupV3WVFXRd5aDUh8/lXBhOdwtfJmmbpOmnon5S0h9JeruZfcTd+3P9ekBoz+Eh/fzx7snrV69fHeNqACTUs5Lms8PPNDhuQdJF4dsknSrpg+4+PSpiTu7+n2bWJWmlpNMlPZKvtRVLGCOxgRgJAAAAIO86W4M4hDm6XskXPtLiplrVVptGx129Q2MaHBlXY13pNDTmo2P44vTlLe6eCu9w914z+29FheMNkm7N5gXN7K2S1kgakfS4ouE4Wfdbl2L+IQrj3+/ZPjmJft2aJTqpg6FzAIrL3S+N4+ua2TGK9tXTFHUKz6soHOhRVBhuztfaionBcwAAAEBhZdv12p1RGCZGQooymjtaG9R1MIpA7e4d0vFLS+dPr6o8vMap6cunZrn/6fTlKfN4ze9I+oyk/60oE/EFM7tiYctDpRpPub5zz/bJ69cwdA5AQqQHvv5CUVH4PQstCptZe/o1XNJz+VthcezvH9ETu3slSTVVpvNOIF8YAAAAyLdwgNpcURJhN3EHHcOTMgbQlVjOcD4Kw+3py0Oz3D9x+6IsXuuHkt4gaZWkRkV/rH4m/dwbzOzybBbk7ufM9KFoKB4qxJ1P9Uy+47K4qVaXv2RFzCsCgMIzs+MVDZM7UdIfHG1onJmtSBeSp9/eIuk6SQ2Sfu7uewqw3ILaHMRInLGqXc31+RqdAAAAAGBCR2uQMTxHYTMsGne00jE8obO1dHOGS+ovKHf/3LSbnpT0CTPbqWja+mck/bToC0NJun7zC5OfX3HOKtXXlE5GCwBky8w+rqmoozPTl+80swvTn9/l7l8PnnKHpBMk3SfpBDO7doaXvc7dt6U/P03Sz83sbkVn93Qrio74DUkrFA2ie1cevpWiC/OFzydfGAAAACiIMBZiT5Ydw2QMT8k4fhVYGJ7oCG6f5f6J2w/m8DW+Lulzks40s1Z3783htVABdh0a1G1PTDW3XbWOGAkAZetySRdNu+3l6Y8JYWH4hPTlOemPmdyhaCisFA3H+z+SzpP0RkVn4QwoevP1y5K+WK77ama+8LIYVwIAAABUro6MjOFhubvM7IjH7SFjeEbh8evpLa0oiXwUhp9MX86WIXxy+nK2DOKjcvchM+uVtFjRcJyy/AMW+XPDPduVnjmn89cu1drlLfEuCAAWyN1fNc/HH/kb2NyP3y7pj+fznHLQ0zusp7v7JEm11aZzjl8c84oAAACAytRSX6Pmumr1j4xrZCylQ4OjWtRUd8TjwpiJMH4i6cJYjVLrGM5HxvDt6cvLzCzj9cysVdIFijqTNi30C5jZqYqKwr2S9i70dVAZxsZTuiEYOnc1Q+cAIHE2BTESZ61erMY64oQAAACAQgmjIbpn6XrNyBimY3hS57SO61KSc2HY3Z+VdIuiU1vfN+3uTynq8P2Wu/dP3Ghmp5nZaeEDzWyNmR0xTtzMlkv6Zvrqd9x9LNc1o7zd8WSPdh2Kftgsba7Tb76YoXMAkDRhvvAG8oUBAACAguo4Sk7u8Ni4DgyMSpKqq0xLmykMT8goDM+R0RyHfA2fe6+kX0n6opldKulxSeslXawoQuIvpz3+8fRleDrsRZK+ZmZ3KRqEs1/ScZJeqyin+F5JH8vTelHGNm4Jhs6du0p1NflofAcAlJNNGfnCFIYBAACAQjpa12sYI7G8pV7VVfNKwKtoYZRET4l1DOelMOzuz5rZuZL+VtEQnddK2iXpC5I+5e4HsniZ+yR9R9EgnbMktSmKjnhE0r9L+id3H8nHelG+ug4O6o4nuyevX83QOQBInD2Hh7R1b3QiUl1Nlc46blG8CwIAAAAqXGZh+Miu1zBGgsFzmRY11aquukoj4yn1Do+pf3hMzfX56tXNTd5WkR5u884sH3vE2wbu/oik38/XelCZbtjywuTQuVecvEzHL22Od0EAgKK7O+gWPue4xWqoJV8YAAAAKKSw67V7hsJw2EW8nMFzGcxMHW312nFgUFKU0bymRArDnIOPsjE2ntIN9wZD5+gWBoBECgvD55MvDAAAABRcx1GGz4XFYjqGj3S0juu4UBhG2bj1ie7Jd6CWt9br1ad3xrwiAEAcMgbPkS8MAAAAFFxn69zD5/YExeKwCIpIRsf1DIX1uFAYRtnYuHlq6NzvnLtKtdX88wWApOk6OKgX9g9Ikhpqq/Sy1e0xrwgAAACofEcbPreHjuE5hcdvpiiOuFBZQ1nYvn9Adz7dI0kyk952HjESAJBEm4IYiXOPX6L6GvKFAQAAgELrCIq9Pb3DcveM+7uDYnEHGcNHCI8fURLAPH17ywua+JnzypOXa/WSpngXBACIRRgjQb4wAAAAUBxNdTVqTQ9MGxlP6eDAaMb93b1Txc4OOoaP0Nk6d8d1XCgMo+SNjqf07/fumLx+9Xq6hQEgqcLBc+QLAwAAAMWT0fXam9n1GhY7yRg+UnjsunvpGAay9l+P7dHevugHTGdbvS49rSPmFQEA4rB9/4C6Dg5KkprqqnXGKvKFAQAAgGKZLWd4aHRchwajDuKaKtOSprqir63UZWYM0zEMZC0cOvfWc1erhqFzAJBIYbfweScsYQgpAAAAUEQdrTPn5GbmC9erqsqKuq5ykBklQccwkJVte/t11zN7JUlVJr11HTESAJBU5AsDAAAA8Qm7Xnt6p4rBYTTCcmIkZtTWWKP6mqgM2z8yrr7hsZhXFKEwjJL27XumuoVfdWqHVi5qjHE1AIC4uHtGx/D55AsDAAAARdXRNnPXa0a+cCuD52ZiZtOiOEqja5jCMErWyFhK3wuHztEtDACJtW3fgHanf3lqra/Ri49ti3lFAAAAQLJ0ts0cJRF+zuC52YVRHKWSM0xhGCXrZ4/u1r7+EUnSMe0NetWpy2NeEQAgLmG38Lo1S8ibBwAAAIqso3Xm4XN7esPCMB3Ds8kYQNdLxzAwp4yhc+cxdA4Akox8YQAAACBeYdE3zBjuyRg+R8fwbDpm6biOE5U2lKStPX2TRYAqiwrDAIBkmp4vvIF8YQAAAKDowqJvd++QUimXlNkx3EHH8KwyM4aJkgBm9e0tU93Cl5zWqWPaGToHAEn1bE+f9vZFvzi1N9bq9GPIFwYAAACKrbGuWm0NNZKk0XHXgYEo/jNj+BwZw7PKyBjupTAMzGhodFzfu29q6Nw1Gxg6BwBJFnYLr1+zRFVVFuNqAAAAgOSaqeuV4XPZyTx2REkAM/rZo7t1YGBUkrRyUaNeeTJD5wAgycgXBgAAAEpDGBXR3TukwZFx9Q6NSZJqq02Lm2rjWlrJCzOauykMAzO7Phg6d9W61aqmMwwAEiuVcm3aun/yOoVhAAAAID6dYc7w4WF1h/nCrQ0yo4Yzm45p3dbuHuNqIhSGUVKe6e7VlueiAkBNlel3zmXoHAAk2VPdvdrfH2WXLWmu0ykdrTGvCAAAAEiujmlxCGG+MIPn5tZaX6OG2qgUOzg6rr7hsZhXRGEYJWbj5u2Tn7/6RZ0ZP3AAAMkT5gtvWEu+MAAAABCnMA5hT+9QZr5wKzWcuZjZjBnNcaIwjJIxNDquG++fGjp39XqGzgFA0oWF4fPXEiMBAAAAxKmjNbOwGRaG6Rg+uswojvhzhikMo2T8+JFdOjQYDZ07bkmTLjxpWcwrAgDEKZVybX6OfGEAAACgVGQMUOsdVk/vcHAfHcNH0zGt4zpuFIZRMjYGQ+fetm41pwsDQMI9tuvw5BuGy1vrdeLylphXBAAAACRbWPztPpwZJdHRSsfw0XRMG94XNwrDKAlP7u7Vvc8fkBQNnbvyHIbOAUDSbdoa5gsvZcIxAAAAELPlrZkdw7sOBRnDdAwfVUZGM4VhILJx8/OTn//mi1dk/KABACQT+cIAAABAaWmorVZ7Y60kaTzlemJ37+R9ZAwfXcbwOaIkAGlwZFzff6Br8jpD5wAA4ynXFvKFAQAAgJITdr1ORL9JmYPVMLOweM7wOUDSzQ/vVO/QmCTphKVNdIUBAPTozkPqHY72hs62ep2wtCnmFQEAAACQZo6MqKuu0qKm2hhWU14yOoaJkgAyh85dte44hs4BAI6IkSBfGAAAACgNHTN0Bne01fM7exY6MjKah+TuMa6GwjBi9tjOw3pw+0FJ0btLV5yzKt4FAQBKwt3B4DliJAAAAIDSMVOWMIPnstNSX6OmumpJ0tBoSofTZ9DHhcIwYrVxSzB07iUrtLSFoHIASLrR8ZTuCfOF1y6LcTUAAAAAQp2tR9ZuOma4DUcys4wietw5wxSGEZv+4THd9MDOyetXr2PoHIBkMbNaM/ugmX3TzB40sxEzczN71xzP+f30Y2b7eM8sz2s0s0+Z2ZNmNmRm3Wb272b2osJ9hwvzSNch9Y+MS5JWLmrU6iWNMa8IAAAAwISZuoPpGM5eWESPO2e4JtavjkS7+aGd6ksPFlq7vFkb1i6JeUUAUHTNkj6f/nyPpN2SVmf53B9KenCG2++dfoOZ1Uv6L0kXpO//QvrrXCnpdWZ2ibtvns/CCynMF95AvjAAAABQUjpmKALPFC+BmYXHr7s33o5hCsOIzcYtU0Pnrl53HH/4A0iiAUmvlfSgu+8ys2slfTLL597k7tdl+dg/U1QU/p6kt7p7SpLM7AZJN0n6hpm9dOL2uG0iXxgAAAAoWZ0zZQzPMJAOM+ssoY5hoiQQi193HdLDOw5JkupqqvTbZzN0DkDyuPuIu//E3XcV6mtY9K7bRLzEx8Lir7v/UNIvJZ0u6aJCrWE+RsZSunfbgcnrFIYBAACA0rJ8poxhOoazFsZu7KmUjGEzW2Vm3zCznWY2bGbbzOzzZrZ4nq+zJP28benX2Zl+XSqHFeT6zVPdwq976TFa3FwX42oAoCydaWYfMrOPm9nb59gnT5R0nKSn3P25Ge7/Sfrykmy+qJndN9OHpNPm/y0c6aEdBzU4GuULH7ekSSsXkS8MACgeM1tqZu8ysx+Y2TNmNmhmh8zsLjP7QzOb8W9oM6tOP+9OMzuQft5WM7vBzE4p9vcBAIVUX1OtxU21GbeRMZy9sIheEVESZnaipF9J6lCUefiEpHWSPijpcjO7wN33zfESE6+zNP06p0i6TdJ3FP2h+U5FGYjnu/vWfKwZ8ekbHtN/PNg1ef3q9QydA4AF+OC06+Nm9nVJH3L38LeLU9OXT83yOk+nL0vij9YwX/j8tXQLAwCK7kpJX5W0S9Ltkl6Q1CnpLZK+Luk1Znalu/vEE8ysRdHfwZcoyv//v5KGJK2U9ApFe+xs+zAAlKXOtgYdGBiduk6URNY6gmPVXSHD5/5RUVH4A+7+pYkbzeyzkj4s6dOaOo11Lv9T0ab5WXf/SPA6H1A0KOcfJV2epzUjJj98sGty2vzJHS069/h5NZUDQNI9J+lPJd0iaYekdkkXSvqMpD+W1Cbp6uDx7enLQ7O83sTti7L54u5+zky3p7uGz87mNeaSURgmRgIAUHxPSXqjpB+F8Utm9glJWyT9tqIi8Y3Bc/5JUVH4Pe7+T9Nf0Mxqp98GAOWuo61BT+zulSTV11SprZExZtkKM5r3xNwxnHOURLpb+DJJ2yR9Zdrdn5TUL+ntZtZ8lNdpkfT29OOvnXb3lyU9L+k3zWxtrmtGfNxdG4MYiavXM3QOQHlLRx/5PD7+LZev5+6/cPcvu/tT7j7g7rvc/buSLpZ0QNJVZvayvHxzRTY0Oq77XiBfGAAQH3e/zd1vnj6Q1d13S/pa+uqrJm43s7MVvSF7w0xF4fRzR2e6HQDKWUeQM9zRVk9tZx46MjKGhxWchFJ0+SjnX5y+vGWGzbPXzP5bUeF4g6Rb53idDZIa06/TO+11Umb2M0l/lP56scVJvLBvQH/308fj+vJlb2TM9ejOw5Kid5TechbR0QDK3rOKThfN1s5CLMLdt5vZjyVdI+mVkh5K3zXREdw+4xOnbj9YiHXNxwMvHNTIWPSrxNplzeSUAQBKzUSBdyy4beIsnW+bWbukN0haLWmfpNvc/Zkirg8AiibseiVGYn5a6mvUUl+jvuExjYyldGhwVIua4pm9lY/CcDbZhZcpioiYqzCctwzE9OmsM8l5MM7hoVH9+JHdub4MJL3+jGPV3sRZVQDKm7tfGvcaAj3py/AsnSfTl7PtnyenL2PPPrx761SMxAa6hQEAJcTMaiT9XvrqT4O7zktfHq/ozeJwA3Mz+6qiyMXxLL5Gwf6OBYB8C5s4wmFqyE5Ha736hqP3Gbt7h2MrDOccJaH8ZRfmNQMRpa2upkrvesWauJcBAJVmffoyPLPmWUWDc04xs5l+8L4mfXlbIReWjXue2z/5OYPnAAAl5u8kvUTSj939Z8HtHenLz0q6Q9KLJLVKerWiPfi9kv66eMsEgOK4+NQO1VZH8RG/+eIVMa+m/ITF9D2H48sZrshk6EIOxlm9uElfuTrn2TqJ95KVbTp+6Zyx0wCAGZjZue5+77TbqiT9haTzJe1V0Mnk7m5mX1M04PV/mdlbJ6KfzOxNiqalPybpF0X6Fmb1jd8/T/e/cEB3P7uPfGEAQMlID0P/iKQnFM3FCU00Wz0h6a1BZ/CtZnaFpPsl/ZmZ/U93H5nr6xR6wCsA5NPqJU365ccu0f7+EZ1+bFvcyyk777pwrd563mp1tjboxcfOlvpXePkoDOcru7AsMhDbm2r1ujOOiXMJAIAKYmYf19QpomemL99pZhemP7/L3b8ePOUeM/u1ogzhLkX74wWKupgGJF3j7oenfZnPSnq9pCskbTazWyUdJ+nK9HP+YPqcgDg01lXrgpOW6YKTlsW9FAAAJElm9n5JX1D0Juql7r5/2kMOpi9vnh4X4e4Pmdlzkk5U1En8kACggqxob9CKdvKFF+LVp3fGvQRJ+SkM5yu7sGwyEAEAyKPLJV007baXpz8mhIXhf5C0TtIlkpZISimKiviKpM+6+xEDWt192Mx+Q9LHJV0l6cOSDku6SdIn3f2xvHwnAABUEDP7kKTPSfq1oqJw9wwPe1LRvnxwlpc5kL5szPf6AADIVT4Kw7enLy8zs6qw48jMWhV1MQ1I2nSU19kkaVDSBWbW6u69wetUKRpgF349AADKnru/ap6P//MFfp0BSX+T/gAAAHMws79QlCv8oKTfcPe9szz054riJV4yw2vUa6rBaVv+VwkAQG5yHj7n7s9KukXSCZLeN+3uTymajP4td++fuNHMTjOzjMmq7t4n6Vvpx1877XXen379n83UCQUAAAAAQD6Y2V8rKgrfp6hTeLaisCTdKGmnpLea2bpp9/21osin2919d0EWCwBADvI1fO69kn4l6YtmdqmkxxVNRr9YUfTDX057/OPpS5t2+yckvUpROP+ZkrYoymJ6k6RuHVl4BgAAAAAgL8zsHZL+VtK4pF9K+oDZ9D9btc3dr5Mkd+83s9+X9J+Sfmlm31c0A2C9pAsV/R37x0VZPAAA85SXwrC7P2tm5yraQC+X9FpJuxSF9H/K3Q/M9fzgdfaZ2fmSPinpzYompe+T9E1Jf+PuO/KxXgAAAAAAZrAmfVkt6UOzPOYXkq6buOLu/5XuFv5rSa9W1CW8W9LXJP2/7r6zUIsFACAX+eoYlrtvl/TOLB97xFuuwX37JX0w/QEAAAAAQFG4+7U6Mtowm+c9JOmKfK8HAIBCyjljGAAAAAAAAABQXigMAwAAAAAAAEDCUBgGAAAAAAAAgIShMAwAAAAAAAAACUNhGAAAAAAAAAAShsIwAAAAAAAAACQMhWEAAAAAAAAASBgKwwAAAAAAAACQMBSGAQAAAAAAACBhKAwDAAAAAAAAQMKYu8e9hqIxs32NjY1LXvSiF8W9FABACXr88cc1ODi4392Xxr2WcsMeCwCYC3vswrHHAgDmkssem7TC8HOS2iRty/GlTktfPpHj6yQVx2/hOHa54fjlJgnH7wRJh919TdwLKTfssSWD47dwHLvccPwWLinH7gSxxy4Ie2xJ4NjlhuO3cBy73CTl+J2gBe6xiSoM54uZ3SdJ7n5O3GspRxy/hePY5YbjlxuOH4qBf2e54fgtHMcuNxy/hePYoVj4t7ZwHLvccPwWjmOXG47f0ZExDAAAAAAAAAAJQ2EYAAAAAAAAABKGwjAAAAAAAAAAJAyFYQAAAAAAAABIGArDAAAAAAAAAJAw5u5xrwEAAAAAAAAAUER0DAMAAAAAAABAwlAYBgAAAAAAAICEoTAMAAAAAAAAAAlDYRgAAAAAAAAAEobCMAAAAAAAAAAkDIVhAAAAAAAAAEgYCsMAAAAAAAAAkDAUhufBzFaZ2TfMbKeZDZvZNjP7vJktjnttpczMlprZu8zsB2b2jJkNmtkhM7vLzP7QzPh3OE9m9rtm5umPd8W9nnJgZpem/w3uTv//u9PMfmZmr417baXOzF5nZreY2Y70/79bzey7ZnZ+3GtD5WCPXRj22Pxjj50/9tiFYX9FsbDHzh/7a/6xv84f++vCscdmz9w97jWUBTM7UdKvJHVI+qGkJyStk3SxpCclXeDu++JbYekys/dI+qqkXZJul/SCpE5Jb5HULulGSVc6/xizYmarJT0iqVpSi6R3u/vX411VaTOz/yXpzyXtkPQTSXslLZd0jqSfu/vHYlxeSTOzv5f0MUn7JN2k6NidJOmNkmok/Z67/1tsC0RFYI9dOPbY/GKPnT/22IVhf0WxsMcuDPtrfrG/zh/768Kxx84PheEsmdnPJF0m6QPu/qXg9s9K+rCkf3L398S1vlJmZpdIapb0I3dPBbevkLRF0mpJV7j7jTEtsWyYmUn6L0lrJH1f0kfFpjonM3u3pH+W9H8l/ZG7j0y7v9bdR2NZXIlL/z/aJalH0hnu3h3cd7Gk2yQ95+5rY1oiKgR77MKxx+YPe+z8sccuDPsriok9dmHYX/OH/XX+2F8Xjj12/jj9IQvpd1kvk7RN0lem3f1JSf2S3m5mzUVeWllw99vc/eZwQ03fvlvS19JXX1X0hZWnD0i6RNI7Ff27wxzMrF7SpxW9w3/EhipJbKhzOl7RPrE53FAlyd1vl9Sr6F1rYMHYY3PDHptX7LHzwB6bE/ZXFAV77MKxv+YV++s8sL/mjD12nigMZ+fi9OUtM2wMvZL+W1KTpA3FXlgFmPiBNhbrKsqAmb1I0t9J+oK73xn3esrEbyj6of99Sal0ztBfmNkHyRbKytOSRiStM7Nl4R1m9kpJrZJ+HsfCUFHYYwuHPTZL7LELwh67cOyvKBb22MJgf80S++uCsL/mhj12nmriXkCZODV9+dQs9z+t6J3YUyTdWpQVVQAzq5H0e+mrP41zLaUufay+pehdw0/EvJxycl76ckjSA5JeEt5pZncqOgWsp9gLKwfuvt/M/kLSZyU9ZmY3KcppOlFRPtN/Sfrj+FaICsEeWwDssdljj10w9tgFYn9FEbHH5hn7a/bYXxeM/TUH7LHzR2E4O+3py0Oz3D9x+6LCL6Wi/J2iH3I/dvefxb2YEvc3ks6SdKG7D8a9mDLSkb78c0mPSXqFpAcV5Vv9g6JfhL8rTgOblbt/3sy2SfqGpHcHdz0j6brpp+cAC8AeWxjssdljj10Y9tgcsL+iSNhj84/9NXvsrwvD/poj9tj5IUoCsTCzD0j6iKKpuG+PeTklzczWK3qH9X+7+91xr6fMTPyMG5P0Rne/y9373P0RSb+laMLrRZySMzsz+5ik70m6TtG7rM2KJuFulXR9elougBLCHps99ticsMfmgP0VKD/sr9ljf80J+2uO2GPnh8JwdibeSW2f5f6J2w8Wfinlz8zeL+kLit79utjd98e8pJKVPv3mXxWd/vXXMS+nHB1MXz7g7tvCO9x9QNLEu/zririmsmFmr5L095L+w93/zN23uvuAu9+v6JeSLkkfMTMmuiIX7LF5xB6bPfbYnB1MX7LHzhP7K4qIPTZP2F+zx/6as4PpS/bXBWCPnT8Kw9l5Mn15yiz3n5y+nC27CWlm9iFJX5L0a0Ub6u54V1TyWhT9u3uRpCEz84kPRZOEJelf0rd9Pq5FlrCJ/3cPznL/gfRlY+GXUpZen768ffod6V9KtijaR84q5qJQcdhj84Q9dt7YY3PDHrtw7K8oFvbYPGB/nTf219ywv+aGPXaeyBjOzsQ/qMvMrCqc6GpmrZIukDQgaVMciysX6QDwv1OUj/Mb7r433hWVhWFJ/2eW+85W9MPsLkWbB6foHOlWSS7p9On/76ZNBPk/V9xllY369OXyWe6fuH2kCGtB5WKPzQP22AVhj80Ne+zCsb+iWNhjc8T+uiDsr7lhf80Ne+x8uTsfWXwoatd3SX867fbPpm//WtxrLOUPRaeQuKR7JS2Jez2V8CHp2vQxfVfcaynlD0k/TB+nD0+7/TJJKUXvuLbHvc5S/JD0O+ljt1vSymn3vSZ9/AYlLY17rXyU9wd7bM7Hjz02/8eUPTa748Qeu7Djxv7KR9E+2GNzOnbsr/k/puyv2R0n9teFHzv22Hl+0DGcvfdK+pWkL5rZpZIel7Re0sWKTr35yxjXVtLM7B2S/lbSuKRfSvqAmU1/2DZ3v67IS0MyvE/Ru9KfNbPXSXpA0UTXNyv6N/kud59tUnPSfU/SzyW9WtLjZvYDRRvsixSdomOSPu7u++JbIioEe+wCscciZuyxC8P+imJij10A9lfEjP114dhj54nCcJbc/VkzO1fR5nC5pNdK2qUogP5T7n5grucn3Jr0ZbWkD83ymF8omhgJ5JW77zCzcyT9jaQ3SnqlpMOSbpb0GXffEuf6Spm7p8zstYp+MXmborD+Jkn7Jf1Y0hfd/ZYYl4gKwR6bE/ZYxIY9dmHYX1FM7LELxv6K2LC/Lhx77PxZup0aAAAAAAAAAJAQVXEvAAAAAAAAAABQXBSGAQAAAAAAACBhKAwDAAAAAAAAQMJQGAYAAAAAAACAhKEwDAAAAAAAAAAJQ2EYAAAAAAAAABKGwjAAAAAAAAAAJAyFYQAAAAAAAABIGArDAAAAAAAAAJAwFIYBAAAAAAAAIGEoDAMAAAAAAABAwlAYBgAAAAAAAICEoTAMAAAAAAAAAAlDYRgAAAAAAAAAEobCMAAAAAAAAAAkDIVhAAAAAAAAAEgYCsMAAAAAAAAAkDD/P+SQaS9BryjFAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 864x288 with 3 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 263,
       "width": 707
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "正在将训练过程转换为gif图, 请耐心等候...: 100%|██████████| 342/342 [00:20<00:00, 56.27it/s]"
     ]
    }
   ],
   "source": [
    "runner.plot_results() # 输出训练结果，可根据该结果对您的机器人进行分析。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "toc-hr-collapsed": false
   },
   "source": [
    "## 2.6 题目二: 实现 Deep QLearning 算法(总分60分)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.6.1 DQN 算法介绍\n",
    "强化学习是一个反复迭代的过程，每一次迭代要解决两个问题：给定一个策略求值函数，和根据值函数来更新策略。而 DQN 算法使用神经网络来近似值函数。([DQN 论文地址](https://files.momodel.cn/Playing%20Atari%20with%20Deep%20Reinforcement%20Learning.pdf))\n",
    "\n",
    "+ **DQN 算法流程**\n",
    "\n",
    "<img src=\"https://imgbed.momodel.cn/20200918101051.png\" width=\"60%\"/>\n",
    "\n",
    "+ **DQN 算法框架图**\n",
    "\n",
    "<img src=\"https://imgbed.momodel.cn/20200918101137.png\" width=\"60%\"/>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "toc-hr-collapsed": false
   },
   "source": [
    "### 2.6.2 完成 DQN 算法"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**ReplayDataSet 类的核心成员方法**\n",
    "\n",
    "+ add(self, state, action_index, reward, next_state, is_terminal) 添加一条训练数据\n",
    "\n",
    "> state: 当前机器人位置\n",
    "\n",
    "> action_index: 选择执行动作的索引\n",
    "\n",
    "> reward： 执行动作获得的回报\n",
    "\n",
    "> next_state：执行动作后机器人的位置\n",
    "\n",
    "> is_terminal：机器人是否到达了终止节点（到达终点或者撞墙）\n",
    "\n",
    "+ random_sample(self, batch_size)：从数据集中随机抽取固定batch_size的数据\n",
    "\n",
    "> batch_size: 整数，不允许超过数据集中数据的个数\n",
    "\n",
    "+ **build_full_view(self, maze)：开启金手指，获取全图视野**\n",
    "\n",
    "> maze: 以 Maze 类实例化的对象"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(array([[0, 1]]), array([[1]], dtype=int8), array([[-10]]), array([[0, 1]]), array([[1]], dtype=int8))\n"
     ]
    }
   ],
   "source": [
    "\"\"\"ReplayDataSet 类的使用\"\"\"\n",
    "\n",
    "from ReplayDataSet import ReplayDataSet\n",
    "\n",
    "test_memory = ReplayDataSet(max_size=1e3) # 初始化并设定最大容量\n",
    "actions = ['u', 'r', 'd', 'l']  \n",
    "test_memory.add((0,1), actions.index(\"r\"), -10, (0,1), 1)  # 添加一条数据（state, action_index, reward, next_state, is_terminal ）\n",
    "print(test_memory.random_sample(1)) # 从中随机抽取一条（因为只有一条数据）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### （1）实现简单的 DQNRobot\n",
    "\n",
    "作业中提供了简单的 DQNRobot 实现，其中依靠简单的两层全连接神经网络决策动作\n",
    "\n",
    "<div align=left>\n",
    "<center><img src=\"https://imgbed.momodel.cn/20201029220521.png\" width=\"241px\"/>\n",
    "</div>\n",
    "\n",
    "+ **该神经网络的输入：机器人当前的位置坐标，输出：执行四个动作（up、right、down、left）的评估分数**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "该部分我们支持 PyTorch 版本和 Keras 版本，大家可以选择自己擅长的深度学习框架！！！ 我们已经实现简单的 DQNRobot 部分，大家可以完善该部分代码！！！"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'hit_wall': 10.0, 'destination': -50.0, 'default': 1.0}\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAroAAAHPCAYAAAC8+nn2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAABYlAAAWJQFJUiTwAAARG0lEQVR4nO3dXaiteUHH8d9fz8z4Qp6yBEtqfLkwAnOyTNHiDKgXJWiYhFQohKVGYRHVpTNERV0kEpVWoBcKvUGGluVLnFOaRErmhfSGzmiR00UyNZnzcvx3sffMnBlmxplz9l7/tX7P5wOLfa72+rGfs9f+7mc/a60x5wwAALR51OoBAABwGoQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlc6sHnA5xhg3JHnT6h2cugtJzq0ewalyjLfBcd4Gx3kbbpxz3rB6xMN1cGd0jyP3+sUzAAC26PrjFjsIBxe6OTqT6zdGAIDdO5cD+qv6QV66cLc551i9AQBgC8YYc/WGR+oQz+gCAMBXJHQBAKgkdAEAqCR0AQCodNBPRqswxkjyxCRPu+R2NslVSa4+/nhVkotJ7kxyx/HHLyX5tySfOb59LnPetev5AAD7SujuyhhfneQFSZ6Re4P26ccfv+oE7uFixvhs7g3fzyT5dJJPJPnHzHlwz5QEALgS49D659KXttjrlxcb4+okz0/ykuPbc7PuUpF/T/LBJB9I8sHMecuiHQDAgTqYBruE0D0pR5cgfEvuDdtzSR6/dNOD+2SOovcDSf46c35x8R4AYM/tbYM9BKF7pcZ4UpIfS/K6JN+4eM3luCPJu5O8JclHXeIAADyQvWuwh0HoXv6Qb03yxiQ/lOSaZTtO1sdyFLx/kDnvWD0GANgfe9Ngj4DQfWR3/ugkL8tR4J7b6X3v1ueTvDXJW13PCwAkQncnlnyRx7gmyRtyFLhP3cl97oc7kvxekl/MnP+8egwAsI7Q3YGdf5HHeFGS30jyzFO/r/11R5JfTfLLnrgGANskdHdgZ1/kMb4+ya8ledWp3cfhuSnJT2bO964eAgDs1iGGrrcAvr8xRsb44SSfisi9v6cmeU/GeGfGeOLqMQAAD8UZ3ft+8sfm6ElYrz7Rz9vps0lekTk/vnoIAHD6nNE9ZGNcm+TDEbkP1zcl+UjGeM3qIQAAD0ToJskYL0jy8STPWT3lwFyT5B0Z480Zw/8lAGCviJMxziV5f5KvXT3lgP1UkreJXQBgn2w7TMZ4cZL3JXn86ikFXpvk7cdvqgEAsNx2Q3eM5yZ5T5LHrp5S5NVJfn31CACAZKuhe/Qaue9O8pjFSxq9IWO8fvUIAIDtvbzYGI9Jcj7J805uFfdzV5IXZ84Lq4cAACfDy4sdhjdF5J62M0nelTGesHoIALBd2wrdMZ6d5GdXz9iIpyT5pdUjAIDt2s6lC0evBvDRJM89hVk8sJnkhZnzo6uHAABXxqUL++0VEbm7NuKsLgCwyDbO6I4xkvxdkm8/pVk8tOdnzr9dPQIAuHzO6O6vF0XkrvTzqwcAANuzldD9kdUDNu7lGePrVo8AALalP3THuCrJ966esXGPSvLS1SMAgG3pD93kXJKzq0eQl68eAABsy1ZCl/WuP35SIADATmwhdJ+5egBJkq9J4jpdAGBnhC675FgAADuzhdB92uoB3MOxAAB2Zguh+5jVA7jHNasHAADbsYXQ9QSo/bGF/28AwJ7YQnj83+oB3ONLqwcAANuxhdC9efUA7nHT6gEAwHZsIXT/afUA7uFYAAA7I3TZlVuT/OfqEQDAdmwhdP9q9QCSJBcy51w9AgDYji2E7vkk/7N6BPmT1QMAgG3pD905b0/yvtUzNm4mee/qEQDAtvSH7pF3rB6wcX+aOV2fCwDs1FZC98+TfHL1iA37ldUDAIDt2UboHj0JSmyt8ZHM+eHVIwCA7RmH9kT4McY9g+ecD//tfcc4k+RjSZ59CrN4cNdnzgurRwAAV+ayG2yhbZzRTZI570ry2iRfXj1lQ35X5AIAq2wndJNkzo8lecvqGRtxS5KfWz0CANiu7Vy6cO8neFySDyf5thOcxX19Ocn3ZM73rx4CAJwMly4cgjm/mOT74u1oT9PPiFwAYLXthW6SzPnZJK9IcufqKYXeHpeHAAB7YJuhmyRzfiRHsXvH6ilF/jDJ63Jo18MAAJW2G7pJMud7k7wsyZdWTynwriQ/mDmdJQcA9sK2QzdJ5vyLJC9N8t+rpxyw30nymuOXcAMA2AtCN0nm/Msk35HkU6unHJg7k/x4ji5XuLh6DADApYTu3eb8lyTPS/JHq6cciM/n6F3Pfss1uQDAPhK6l5rztiQ/kOT1SW5bvGaf/XGS6zLn36weAgDwYITu/c05M+fbkjwryXtWz9kz/5HkVUm+P3PesnoMAMBDEboPZs6bMufLkrw8yc2r5yx2Mcmbk3xz5vx9lyoAAIdge28BfHl3+rgkP53kJ5I8eSf3uR8u5ugyhV/InJ9cPQYAWOcQ3wJY6D6yO786ySuTvDHJd+70vnfrC0l+O8lvHr+LHACwcUJ3B/bmizzG83MUvK9McmbZjpP1qRy9fe87M+cXV48BAPbH3jTYIyB0r9QYT8nRa8n+aJInLV5zOb6c5M9yFLgfcv0tAPBA9q7BHgahe1LGeHSS5yR5yfHthUmuWrrpwf1rkg8c385nzi8s3gMA7Lm9bbCHIHRPyxiPT/LduTd8n7VwzX8l+VDujts5b1q4BQA4QAfTYJcQursyxpOTfFeSZyR52vHt6Umuzcmd+b0lyWcuuX06yT8k+Xtv0QsAXIlDbDChu9rRJQ/fkPvG7xNyFL9XX/LxriR3Jrnj+OPtST6Xe6P2psz5v7ueDwBswyE2mNAFAOArOsQG885oAABUEroAAFQSugAAVBK6AABUEroAAFQ6s3rAlbj02X9UujDnvH71CE7PGON8knOrd3DqfC8DSxxi6F5Icl2Ss4t3APDwXDfGOC92ex3/0pokcZyrXVg94JE6xNB19gfgsJyNx+52ju82HNxx9oYRAJwaj9nb4DhvwyEeZ09GAwCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoJLQBQCgktAFAKCS0AUAoNKZ1QOA7RpjnE9ybvUOdmOMMVdvALblEEP3wuoBnL7jAEqSzDmvX7cEOAEXkzx69QhO3cUkt60ewak6uAYbcx7WL9iXnhGYc46VWzg9jvM2OKO7GbcmObt6BLvhMbvXIf5sFrrsJccZ4HB4zN6GQzzOnowGAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBAJaELAEAloQsAQCWhCwBApTOrB8BXMsaYqzdwam5Ncnb1CE6d4wwscYihe2H1AHbiQpLr4ocjwKG4mOS21SM4VQfXYGPOwzpZdunZvTnnWLmF0zXGOJ/k3OodnCpn+rbBcd4QP5t7HWKDCV0A4Ir42bwNh3icPRkNAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASmdWD4AHM8Y4n+Tc6h2cqgtzzutXj+B0+V7eljHGXL0B7naIoXth9QDgxFw3xjgvdqHASCJx+12T5PbDabFDDF1nBaDH2fiehg5XJ7l99QhO3dExPpjH7THnYf36demfROacY+UW4PL5XoYe40aXK2zCDff+81Aetz0ZDQCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpCFwCASkIXAIBKQhcAgEpnVg+4EmOMuXoDp+pCknOrR3D6fC/X8728BdcmuXn1CLivQzyje2OOHjSBw3fr6gEAPALXJjlqsYNwcKE757whyfnFM4CT8YnVA4ATcrOTUJtwc248brGDMOb0F0MAAPoc3BldAAB4OIQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlYQuAACVhC4AAJWELgAAlf4fUw8M6YhyAc0AAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 231,
       "width": 349
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Maze of size (5, 5)\n",
      "action: r reward: 1.0\n",
      "action: d reward: 1.0\n",
      "action: l reward: 1.0\n",
      "action: d reward: 1.0\n",
      "action: d reward: 1.0\n",
      "action: r reward: 1.0\n",
      "action: r reward: 1.0\n",
      "action: d reward: 1.0\n",
      "action: r reward: 1.0\n",
      "action: r reward: -50.0\n",
      "success\n"
     ]
    }
   ],
   "source": [
    "from torch_py.MinDQNRobot import MinDQNRobot as TorchRobot # PyTorch版本\n",
    "# from keras_py.MinDQNRobot import MinDQNRobot as KerasRobot # Keras版本\n",
    "# from Robot import Robot as TorchRobot\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "from Maze import Maze\n",
    "from Runner import Runner\n",
    "import os\n",
    "\n",
    "os.environ[\"KMP_DUPLICATE_LIB_OK\"] = \"TRUE\"  # 允许重复载入lib文件\n",
    "\n",
    "maze = Maze(maze_size=5) \n",
    "\n",
    "\n",
    "\"\"\"选择keras版本或者torch版本的机器人, MinRobot是尽量选择reward值最小的动作，对象初始化过程中修改了maze的reward参数\"\"\"\n",
    "# robot = KerasRobot(maze=maze)\n",
    "robot = TorchRobot(maze=maze)\n",
    "\n",
    "print(robot.maze.reward) # 输出最小值选择策略的reward值\n",
    "\n",
    "\"\"\"开启金手指，获取全图视野\"\"\"\n",
    "robot.memory.build_full_view(maze=maze) # \n",
    "\n",
    "\"\"\"training by runner\"\"\"\n",
    "runner = Runner(robot=robot)\n",
    "runner.run_training(training_epoch=10, training_per_epoch=75)\n",
    "\n",
    "\"\"\"Test Robot\"\"\"\n",
    "robot.reset()\n",
    "print(maze)\n",
    "for _ in range(25):\n",
    "    a, r = robot.test_update()\n",
    "    print(\"action:\", a, \"reward:\", r)\n",
    "    if r == maze.reward[\"destination\"]:\n",
    "        print(\"success\")\n",
    "        break"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### （2）实现你自己的 DQNRobot\n",
    "\n",
    " + **题目要求:** 编程实现 DQN 算法在机器人自动走迷宫中的应用\n",
    " + **输入:** 由 Maze 类实例化的对象 maze\n",
    " + **要求不可更改的成员方法：**train_update()、test_update() **注：不能修改该方法的输入输出及方法名称，测试评分会调用这两个方法**。\n",
    " + **补充1:**若要自定义的参数变量，在 \\_\\_init\\_\\_() 中以 `self.xxx = xxx` 创建即可\n",
    " + **补充2:**实现你自己的DQNRobot时，要求继承 QRobot 类，QRobot 类包含了某些固定的方法如reset(重置机器人位置),sense_state(获取机器人当前位置).."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {
    "deletable": false,
    "select": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import random\n",
    "\n",
    "import torch\n",
    "import torch.nn.functional as F\n",
    "from torch import optim\n",
    "\n",
    "from QRobot import QRobot\n",
    "from Maze import Maze\n",
    "from ReplayDataSet import ReplayDataSet\n",
    "from torch_py.QNetwork import QNetwork\n",
    "\n",
    "\n",
    "class MinDQNRobot(QRobot):\n",
    "    valid_action = ['u', 'r', 'd', 'l']\n",
    "\n",
    "    ''' QLearning parameters'''\n",
    "    epsilon0 = 0.5  # 初始贪心算法探索概率\n",
    "    gamma = 0.94  # 公式中的 γ\n",
    "\n",
    "    EveryUpdate = 1  # the interval of target model's updating\n",
    "\n",
    "    \"\"\"some parameters of neural network\"\"\"\n",
    "    target_model = None\n",
    "    eval_model = None\n",
    "    batch_size = 32\n",
    "    learning_rate = 1e-2\n",
    "    TAU = 1e-3\n",
    "    step = 1  # 记录训练的步数\n",
    "\n",
    "    \"\"\"setting the device to train network\"\"\"\n",
    "    device = torch.device(\"cuda:0\") if torch.cuda.is_available() else torch.device(\"cpu\")\n",
    "\n",
    "    def __init__(self, maze):\n",
    "        \"\"\"\n",
    "        初始化 Robot 类\n",
    "        :param maze:迷宫对象\n",
    "        \"\"\"\n",
    "        super(MinDQNRobot, self).__init__(maze)\n",
    "        maze.set_reward(reward={\n",
    "            \"hit_wall\": 10.,\n",
    "            \"destination\": -50.,\n",
    "            \"default\": 1.,\n",
    "        })\n",
    "        self.maze = maze\n",
    "        self.maze_size = maze.maze_size\n",
    "\n",
    "        \"\"\"build network\"\"\"\n",
    "        self.target_model = None\n",
    "        self.eval_model = None\n",
    "        self._build_network()\n",
    "\n",
    "        \"\"\"create the memory to store data\"\"\"\n",
    "        max_size = max(self.maze_size ** 2 * 3, 1e4)\n",
    "        self.memory = ReplayDataSet(max_size=max_size)\n",
    "\n",
    "    def _build_network(self):\n",
    "        seed = 0\n",
    "        random.seed(seed)\n",
    "\n",
    "        \"\"\"build target model\"\"\"\n",
    "        self.target_model = QNetwork(state_size=2, action_size=4, seed=seed).to(self.device)\n",
    "\n",
    "        \"\"\"build eval model\"\"\"\n",
    "        self.eval_model = QNetwork(state_size=2, action_size=4, seed=seed).to(self.device)\n",
    "\n",
    "        \"\"\"build the optimizer\"\"\"\n",
    "        self.optimizer = optim.Adam(self.eval_model.parameters(), lr=self.learning_rate)\n",
    "\n",
    "    def target_replace_op(self):\n",
    "        \"\"\"\n",
    "            Soft update the target model parameters.\n",
    "            θ_target = τ*θ_local + (1 - τ)*θ_target\n",
    "        \"\"\"\n",
    "\n",
    "        # for target_param, eval_param in zip(self.target_model.parameters(), self.eval_model.parameters()):\n",
    "        #     target_param.data.copy_(self.TAU * eval_param.data + (1.0 - self.TAU) * target_param.data)\n",
    "\n",
    "        \"\"\" replace the whole parameters\"\"\"\n",
    "        self.target_model.load_state_dict(self.eval_model.state_dict())\n",
    "\n",
    "    def _choose_action(self, state):\n",
    "        state = np.array(state)\n",
    "        state = torch.from_numpy(state).float().to(self.device)\n",
    "        if random.random() < self.epsilon:\n",
    "            action = random.choice(self.valid_action)\n",
    "        else:\n",
    "            self.eval_model.eval()\n",
    "            with torch.no_grad():\n",
    "                q_next = self.eval_model(state).cpu().data.numpy()  # use target model choose action\n",
    "            self.eval_model.train()\n",
    "\n",
    "            action = self.valid_action[np.argmin(q_next).item()]\n",
    "        return action\n",
    "\n",
    "    def _learn(self, batch: int = 16):\n",
    "        if len(self.memory) < batch:\n",
    "            print(\"the memory data is not enough\")\n",
    "            return\n",
    "        state, action_index, reward, next_state, is_terminal = self.memory.random_sample(batch)\n",
    "\n",
    "        \"\"\" convert the data to tensor type\"\"\"\n",
    "        state = torch.from_numpy(state).float().to(self.device)\n",
    "        action_index = torch.from_numpy(action_index).long().to(self.device)\n",
    "        reward = torch.from_numpy(reward).float().to(self.device)\n",
    "        next_state = torch.from_numpy(next_state).float().to(self.device)\n",
    "        is_terminal = torch.from_numpy(is_terminal).int().to(self.device)\n",
    "\n",
    "        self.eval_model.train()\n",
    "        self.target_model.eval()\n",
    "\n",
    "        \"\"\"Get max predicted Q values (for next states) from target model\"\"\"\n",
    "        Q_targets_next = self.target_model(next_state).detach().min(1)[0].unsqueeze(1)\n",
    "\n",
    "        \"\"\"Compute Q targets for current states\"\"\"\n",
    "        Q_targets = reward + self.gamma * Q_targets_next * (torch.ones_like(is_terminal) - is_terminal)\n",
    "\n",
    "        \"\"\"Get expected Q values from local model\"\"\"\n",
    "        self.optimizer.zero_grad()\n",
    "        Q_expected = self.eval_model(state).gather(dim=1, index=action_index)\n",
    "\n",
    "        \"\"\"Compute loss\"\"\"\n",
    "        loss = F.mse_loss(Q_expected, Q_targets)\n",
    "        loss_item = loss.item()\n",
    "\n",
    "        \"\"\" Minimize the loss\"\"\"\n",
    "        loss.backward()\n",
    "        self.optimizer.step()\n",
    "\n",
    "        \"\"\"copy the weights of eval_model to the target_model\"\"\"\n",
    "        self.target_replace_op()\n",
    "        return loss_item\n",
    "\n",
    "    def train_update(self):\n",
    "        state = self.sense_state()\n",
    "        action = self._choose_action(state)\n",
    "        reward = self.maze.move_robot(action)\n",
    "        next_state = self.sense_state()\n",
    "        is_terminal = 1 if next_state == self.maze.destination or next_state == state else 0\n",
    "\n",
    "        self.memory.add(state, self.valid_action.index(action), reward, next_state, is_terminal)\n",
    "\n",
    "        \"\"\"--间隔一段时间更新target network权重--\"\"\"\n",
    "        if self.step % self.EveryUpdate == 0:\n",
    "            self._learn(batch=32)\n",
    "\n",
    "        \"\"\"---update the step and epsilon---\"\"\"\n",
    "        self.step += 1\n",
    "        self.epsilon = max(0.01, self.epsilon * 0.995)\n",
    "\n",
    "        return action, reward\n",
    "\n",
    "    def test_update(self):\n",
    "        state = np.array(self.sense_state(), dtype=np.int16)\n",
    "        state = torch.from_numpy(state).float().to(self.device)\n",
    "\n",
    "        self.eval_model.eval()\n",
    "        with torch.no_grad():\n",
    "            q_value = self.eval_model(state).cpu().data.numpy()\n",
    "\n",
    "        action = self.valid_action[np.argmin(q_value).item()]\n",
    "        reward = self.maze.move_robot(action)\n",
    "        return action, reward\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {},
   "outputs": [],
   "source": [
    "from QRobot import QRobot\n",
    "\n",
    "\"\"\"\n",
    "题目要求:  编程实现 DQN 算法在机器人自动走迷宫中的应用\n",
    "输入: 由 Maze 类实例化的对象 maze\n",
    "必须完成的成员方法：train_update()、test_update()\n",
    "补充：如果想要自定义的参数变量，在 \\_\\_init\\_\\_() 中以 `self.xxx = xxx` 创建即可\n",
    "\"\"\"\n",
    "\n",
    "\n",
    "import numpy as np\n",
    "import random\n",
    "\n",
    "import torch\n",
    "import torch.nn.functional as F\n",
    "from torch import optim\n",
    "\n",
    "from QRobot import QRobot\n",
    "from Maze import Maze\n",
    "from ReplayDataSet import ReplayDataSet\n",
    "from torch_py.QNetwork import QNetwork\n",
    "\n",
    "\n",
    "class Robot(QRobot):\n",
    "    valid_action = ['u', 'r', 'd', 'l']\n",
    "\n",
    "    ''' QLearning parameters'''\n",
    "    epsilon0 = 0.5  # 初始贪心算法探索概率\n",
    "    gamma = 0.94  # 公式中的 γ\n",
    "\n",
    "    EveryUpdate = 1  # the interval of target model's updating\n",
    "\n",
    "    \"\"\"some parameters of neural network\"\"\"\n",
    "    target_model = None\n",
    "    eval_model = None\n",
    "    batch_size = 32\n",
    "    learning_rate = 1e-2\n",
    "    TAU = 1e-3\n",
    "    step = 1  # 记录训练的步数\n",
    "\n",
    "    \"\"\"setting the device to train network\"\"\"\n",
    "    device = torch.device(\"cuda:0\") if torch.cuda.is_available() else torch.device(\"cpu\")\n",
    "\n",
    "    def __init__(self, maze):\n",
    "        \"\"\"\n",
    "        初始化 Robot 类\n",
    "        :param maze:迷宫对象\n",
    "        \"\"\"\n",
    "        super(Robot, self).__init__(maze)\n",
    "        maze.set_reward(reward={\n",
    "            \"hit_wall\": -10.,\n",
    "            \"destination\": 50.,\n",
    "            \"default\": -1.,\n",
    "        })\n",
    "        self.maze = maze\n",
    "        self.maze_size = maze.maze_size\n",
    "\n",
    "        \"\"\"build network\"\"\"\n",
    "        self.target_model = None\n",
    "        self.eval_model = None\n",
    "        self._build_network()\n",
    "\n",
    "        \"\"\"create the memory to store data\"\"\"\n",
    "        max_size = max(self.maze_size ** 2 * 3, 1e4)\n",
    "        self.memory = ReplayDataSet(max_size=max_size)\n",
    "\n",
    "    def _build_network(self):\n",
    "        seed = 0\n",
    "        random.seed(seed)\n",
    "\n",
    "        \"\"\"build target model\"\"\"\n",
    "        self.target_model = QNetwork(state_size=2, action_size=4, seed=seed).to(self.device)\n",
    "\n",
    "        \"\"\"build eval model\"\"\"\n",
    "        self.eval_model = QNetwork(state_size=2, action_size=4, seed=seed).to(self.device)\n",
    "\n",
    "        \"\"\"build the optimizer\"\"\"\n",
    "        self.optimizer = optim.Adam(self.eval_model.parameters(), lr=self.learning_rate)\n",
    "\n",
    "    def target_replace_op(self):\n",
    "        \"\"\"\n",
    "            Soft update the target model parameters.\n",
    "            θ_target = τ*θ_local + (1 - τ)*θ_target\n",
    "        \"\"\"\n",
    "\n",
    "        # for target_param, eval_param in zip(self.target_model.parameters(), self.eval_model.parameters()):\n",
    "        #     target_param.data.copy_(self.TAU * eval_param.data + (1.0 - self.TAU) * target_param.data)\n",
    "\n",
    "        \"\"\" replace the whole parameters\"\"\"\n",
    "        self.target_model.load_state_dict(self.eval_model.state_dict())\n",
    "\n",
    "    def _choose_action(self, state):\n",
    "        state = np.array(state)\n",
    "        state = torch.from_numpy(state).float().to(self.device)\n",
    "        if random.random() < self.epsilon:\n",
    "            action = random.choice(self.valid_action)\n",
    "        else:\n",
    "            self.eval_model.eval()\n",
    "            with torch.no_grad():\n",
    "                q_next = self.eval_model(state).cpu().data.numpy()  # use target model choose action\n",
    "            self.eval_model.train()\n",
    "\n",
    "            action = self.valid_action[np.argmax(q_next).item()]\n",
    "        return action\n",
    "\n",
    "    def _learn(self, batch: int = 16):\n",
    "        if len(self.memory) < batch:\n",
    "            print(\"the memory data is not enough\")\n",
    "            return\n",
    "        state, action_index, reward, next_state, is_terminal = self.memory.random_sample(batch)\n",
    "\n",
    "        \"\"\" convert the data to tensor type\"\"\"\n",
    "        state = torch.from_numpy(state).float().to(self.device)\n",
    "        action_index = torch.from_numpy(action_index).long().to(self.device)\n",
    "        reward = torch.from_numpy(reward).float().to(self.device)\n",
    "        next_state = torch.from_numpy(next_state).float().to(self.device)\n",
    "        is_terminal = torch.from_numpy(is_terminal).int().to(self.device)\n",
    "\n",
    "        self.eval_model.train()\n",
    "        self.target_model.eval()\n",
    "\n",
    "        \"\"\"Get max predicted Q values (for next states) from target model\"\"\"\n",
    "        Q_targets_next = self.target_model(next_state).detach().max(1)[0].unsqueeze(1)\n",
    "\n",
    "        \"\"\"Compute Q targets for current states\"\"\"\n",
    "        Q_targets = reward + self.gamma * Q_targets_next * (torch.ones_like(is_terminal) - is_terminal)\n",
    "\n",
    "        \"\"\"Get expected Q values from local model\"\"\"\n",
    "        self.optimizer.zero_grad()\n",
    "        Q_expected = self.eval_model(state).gather(dim=1, index=action_index)\n",
    "\n",
    "        \"\"\"Compute loss\"\"\"\n",
    "        loss = F.mse_loss(Q_expected, Q_targets)\n",
    "        loss_item = loss.item()\n",
    "\n",
    "        \"\"\" Minimize the loss\"\"\"\n",
    "        loss.backward()\n",
    "        self.optimizer.step()\n",
    "\n",
    "        \"\"\"copy the weights of eval_model to the target_model\"\"\"\n",
    "        self.target_replace_op()\n",
    "        return loss_item\n",
    "\n",
    "\n",
    "    def train_update(self):\n",
    "        \"\"\" \n",
    "        以训练状态选择动作并更新Deep Q network的相关参数\n",
    "        :return :action, reward 如：\"u\", -1\n",
    "        \"\"\"\n",
    "\n",
    "        action, reward = \"u\", -1.0\n",
    "\n",
    "        # -----------------请实现你的算法代码--------------------------------------\n",
    "        state = self.sense_state()\n",
    "        action = self._choose_action(state)\n",
    "        reward = self.maze.move_robot(action)\n",
    "        next_state = self.sense_state()\n",
    "        is_terminal = 1 if next_state == self.maze.destination or next_state == state else 0\n",
    "\n",
    "        self.memory.add(state, self.valid_action.index(action), reward, next_state, is_terminal)\n",
    "\n",
    "        \"\"\"--间隔一段时间更新target network权重--\"\"\"\n",
    "        if self.step % self.EveryUpdate == 0:\n",
    "            self._learn(batch=32)\n",
    "\n",
    "        \"\"\"---update the step and epsilon---\"\"\"\n",
    "        self.step += 1\n",
    "        self.epsilon = max(0.01, self.epsilon * 0.995)\n",
    "        # -----------------------------------------------------------------------\n",
    "\n",
    "        return action, reward\n",
    "\n",
    "    def test_update(self):\n",
    "        \"\"\"\n",
    "        以测试状态选择动作并更新Deep Q network的相关参数\n",
    "        :return :action, reward 如：\"u\", -1\n",
    "        \"\"\"\n",
    "\n",
    "        action, reward = \"u\", -1.0\n",
    "\n",
    "        # -----------------请实现你的算法代码--------------------------------------\n",
    "        state = np.array(self.sense_state(), dtype=np.int16)\n",
    "        state = torch.from_numpy(state).float().to(self.device)\n",
    "\n",
    "        self.eval_model.eval()\n",
    "        with torch.no_grad():\n",
    "            q_value = self.eval_model(state).cpu().data.numpy()\n",
    "\n",
    "        action = self.valid_action[np.argmax(q_value).item()]\n",
    "        reward = self.maze.move_robot(action)\n",
    "        # -----------------------------------------------------------------------\n",
    "\n",
    "        return action, reward\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### （3）测试您的 DQN 算法\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n",
      "the memory data is not enough\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "正在将训练过程转换为gif图, 请耐心等候...: 100%|██████████| 370/370 [00:22<00:00, 58.19it/s]"
     ]
    }
   ],
   "source": [
    "from QRobot import QRobot\n",
    "from Maze import Maze\n",
    "from Runner import Runner\n",
    "from Robot import Robot\n",
    "\n",
    "\"\"\"  Deep Qlearning 算法相关参数： \"\"\"\n",
    "\n",
    "epoch = 10  # 训练轮数\n",
    "maze_size = 5  # 迷宫size\n",
    "training_per_epoch=int(maze_size * maze_size * 1.5)\n",
    "\n",
    "\"\"\" 使用 DQN 算法训练 \"\"\"\n",
    "\n",
    "g = Maze(maze_size=maze_size)\n",
    "# r = MinDQNRobot(g)\n",
    "r = Robot(g)\n",
    "runner = Runner(r)\n",
    "runner.run_training(epoch, training_per_epoch)\n",
    "\n",
    "# 生成训练过程的gif图, 建议下载到本地查看；也可以注释该行代码，加快运行速度。\n",
    "runner.generate_gif(filename=\"results/dqn_size10.gif\")\n",
    "runner.plot_results()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.6.3 作业测试与提交"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 经过 `2.3` 与 `2.6` 分别测试使用基础算法、DQN算法实现机器人走出迷宫！\n",
    "- 测试完成之后，点击左侧 `提交作业` 的标签中，把整个 Notebook 目标 cell 转化为 main.py 文件进行`系统测试`。\n",
    "- 平台测试时请记得勾选 main.py 文件需要依赖的其它文件等。\n",
    "- 通过测试就可以**提交作业**。\n",
    "-  提交作业时请记得提交勾选 **『程序报告.docx』**或者 **『程序报告.pdf』**。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**最后，祝愿您不仅能从中收获到满满的知识，而且收获到一个满意分数！**"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
